Innovate QA Meetup Group

AI Governance Needs a Test Plan: Turning Principles into Evidence

Thu, Oct 15 · 4:00 PM PDT

·

Online

Online

Speaker: Tanvi Mittal | LinkedIn

Topic: AI Governance Needs a Test Plan: Turning Principles into Evidence

Agenda:
🕛 4:00–4:05 PM PST — Welcome
🕛 4:05–4:55 PM PST — Tanvi Mittal | LinkedIn - AI Governance Needs a Test Plan: Turning Principles into Evidence
🕛 4:55–5:00 PM PST — Q&A & Closing

Summary :
AI governance is increasingly shaped by frameworks such as the EU AI Act, the NIST AI Risk Management Framework, and ISO/IEC 42001. These frameworks define what AI systems need to demonstrate around safety, fairness, transparency, accountability, and risk management, but they often leave an important question unanswered: how do you actually prove it?

In this session, Tanvi Mittal will connect AI governance principles to practical Quality Engineering and testing practices. The discussion will explore how requirements around risk management, robustness, human oversight, and continuous monitoring can be translated into concrete QA artifacts and evidence.

Tanvi will introduce PROBE — Probabilistic Baseline, Red Team It, Observe in Production, Behavioral Contracts, and Evals Not Assertions — and show how each part can be applied to testing non-deterministic AI systems.

The session will also explore how QA teams can create practical evidence for AI behavior, including distributions, adversarial test suites, production monitoring, behavioral contracts, and evaluation gates.

Takeaways :
What attendees will learn:

1. How to translate AI governance principles into practical QA and testing activities
2. How probabilistic baselines can help establish expected AI system behavior
3. How to use red-team testing to identify risks and weaknesses in AI systems
4. How production observation and continuous monitoring can provide evidence of AI behavior
5. How behavioral contracts can turn governance expectations into testable requirements
6. How evaluation gates can be used to assess non-deterministic AI systems beyond traditional assertions
7. How QA teams can provide evidence of AI system behavior to auditors and executives

🎙️ Speaker Opportunities at Our Meetups

Have expertise in QA, leadership, or tech skills you'd love to share? We're always looking for engaging speakers for our monthly meetups. This is a fantastic platform to showcase your knowledge, connect with the community, and inspire others. If you're interested submit your topic here — we’d love to hear from you!

How to evaluate or test the product powered by large language models

Tue, Oct 20 · 4:00 PM PDT

·

Online

Online

Speaker: Narendra Singh Panwar | LinkedIn

Topic: How to Evaluate or Test Products Powered by Large Language Models

Agenda:
🕛 4:00–4:05 PM PDT — Welcome
🕛 4:05–4:55 PM PDT — Narendra Singh Panwar | LinkedIn - How to Evaluate or Test Products Powered by Large Language Models
🕛 4:55–5:00 PM PDT — Q&A & Closing

Summary :
LLMs and AI agents are powering the next generation of products. While the speed of development has increased significantly, evaluating the end-to-end quality of these products remains a challenge.

In this session, Narendra Singh Panwar will share practical learnings and techniques from testing consumer-facing products powered by LLMs and AI agents. The discussion will explore how traditional QA practices need to evolve when testing systems that are non-deterministic and capable of making decisions, generating responses, and completing actions.

The session will cover how teams can evaluate the non-deterministic nature of AI systems, verify factual accuracy, and determine where AI-assisted evaluation can help. Narendra will also discuss the use of LLM-as-a-Judge and where human evaluation is still necessary to assess the quality of AI-powered experiences.

The discussion will also look at AI-powered test automation and continuous customer experience monitoring as ways to support quality evaluation as AI products evolve.

Takeaways :
What attendees will learn:

1. How to evaluate the non-deterministic behavior of LLM-powered products
2. Approaches for fact-checking and evaluating AI-generated responses
3. Where AI and LLM-as-a-Judge can be applied in quality evaluation
4. Where human evaluation is still necessary
5. How AI-powered test automation can support testing of AI products
6. How continuous CX monitoring can help evaluate end-to-end quality
7. How traditional QA practices can evolve for products powered by LLMs and AI agents

🎙️ Speaker Opportunities at Our Meetups

Have expertise in QA, leadership, or tech skills you'd love to share? We're always looking for engaging speakers for our monthly meetups. This is a fantastic platform to showcase your knowledge, connect with the community, and inspire others. If you're interested submit your topic here — we’d love to hear from you!

From Ai-Generated Tests to Real Quality

Thu, Oct 22 · 6:00 PM PDT

1515 Anderson St, 1515 Anderson St, Vancouver, BC V6H 3R5, Canada, Vancouver, BC, CA

Speaker: Tatyana Arbouzova | LinkedIn

Topic: From Co-Pilot to Colleague, at Enterprise Scale

Innovate QA community is going to Vancouver, Canada and partnering with Ministry of Testing local chapter on our October meetup. Hope to see you there.

Agenda:
🕕 6:00–6:45 PM PST — Networking
🕕 6:45–7:30 PM PST — Tatyana Arbouzova - From Co-Pilot to Colleague, at Enterprise Scale
🕕 7:30–8:00 PM PST — Networking

Summary :
AI can generate tests. But can it deliver quality?

For most teams, the honest answer is “not yet.”

AI co-pilots have made individual testing tasks dramatically faster—from generating tests based on requirements, tickets, designs, and API specifications to running tests across web, mobile, and API and investigating failures down to root cause.

But as enterprise teams adopt these tools, a gap becomes clear: every step may be faster, yet a person still has to start every one. Someone has to spot the risky PR, choose the right tests, triage failures, and follow up on the fix.
We automated the tasks. We hadn't automated the initiative.

In this session, Tatyana Arbouzova will explore the journey from AI as a co-pilot that waits for human direction to AI acting more like a colleague that can initiate and drive quality activities.
The session will look at what enterprise AI co-pilots do well, where they still fall short, and what enterprise teams have learned as they move toward more autonomous quality engineering.

You'll also see a live walkthrough of this shift in action: a PR opens, impact analysis runs automatically, tests are generated and executed from API through UI, a regression is identified, and the root cause reaches GitHub or Slack before anyone goes looking for it.

You'll leave with a simple question to take back to your own team:
Where does your AI still wait for a human to start the work?

Then come find us at the ContextQA booth to see how far that answer can go.

Takeaways:
What attendees will learn:

1. A clear model for the two stages of AI in quality engineering: the co-pilot that speeds up human work and the colleague that initiates it.
2. Real lessons from enterprise teams about where AI-assisted testing still depends on people to start, prioritize, and follow through.
3. An end-to-end view of autonomous quality in action, from PR to impact analysis to tests to root cause, running on an enterprise-grade platform.
4. A practical way to identify the next step in your own team's AI adoption without replacing what already works.

🎙️ Speaker Opportunities at Our Meetups

Have expertise in QA, leadership, or tech skills you'd love to share? We're always looking for engaging speakers for our monthly meetups. This is a fantastic platform to showcase your knowledge, connect with the community, and inspire others. If you're interested submit your topic here — we’d love to hear from you!