Testing AI Systems and Testing with AI

AI is changing both how software testing is performed and what testing professionals are expected to evaluate. Testing with AI and Testing AI Systems addresses both sides of this transformation: using AI to improve testing work and applying specialized approaches to test AI-powered systems.

The first part, Testing with AI, helps learners build essential AI literacy and understand the role of machine learning, generative AI, large language models, and data in modern software products. Learners develop practical co-piloting and prompt-engineering skills, then apply generative AI across test analysis, test design, exploratory testing, documentation, reporting, and other everyday testing activities.

A major part of the course introduces 22 structured ways AI can assist manual and exploratory testing. These practices help learners use AI to identify ambiguity, generate test ideas, analyze coverage, create exploratory charters, prioritize risks, improve bug reports, design test data, and support professional communication. The course also introduces custom AI-powered solutions that can be adapted to specific testing workflows.

The second part, Testing AI Systems, focuses on the characteristics and risks that distinguish AI software from traditional deterministic systems. Learners examine probabilistic outputs, data dependency, bias, model drift, hallucinations, adversarial behavior, explainability, instruction fidelity, output consistency, latency, scalability, and Human-in-the-Loop evaluation.

Practical testing methods, tools, demonstrations, and exercises help learners evaluate AI systems more systematically. The emphasis remains on human accountability: AI may assist with analysis and execution, but testers remain responsible for context, evaluation criteria, quality decisions, and approval.

By completing the course, learners will be able to:

• Understand essential AI, GenAI, and LLM concepts.
• Collaborate with AI through structured prompting and iterative refinement.
• Apply AI assistance across practical testing workflows.
• Recognize the limitations and common failure patterns of AI output.
• Distinguish traditional software testing from AI systems testing.
• Evaluate bias, fairness, robustness, explainability, and consistency.
• Apply black-box exploratory methods to AI-powered applications.
• Use appropriate methods and tools for continuous AI evaluation.
• Maintain human oversight when using or testing AI systems.