AI - Episode 10 - Evals, Hallucinations & Keeping the Robot From Ruining Tuesday

The demo worked. Unfortunately, that’s not a test strategy. We try to break Support Assistant with missing policies, conflicting documents, unavailable services, malicious instructions, and unauthorized refund requests. Learn how ordinary tests, AI evaluations, human review, monitoring, and traces work together—and why a convincing answer or impressive average score can hide a serious failure.