VIDEO COURSE · COMING SOON
Testing AI
Engineering Confidence in Non-Deterministic Systems
Learn to test systems that don't give the same answer twice. Practical lessons, worked examples, and exercises with Jason Arbon.
Tell me when the videos are readyVideo lessons are being planned. No videos or paid enrollment are available yet.
What you'll explore
- Make quality measurableDefine the behaviors that matter and build a useful scoring rubric.
- Sample, don't guessUse repeated runs to understand variation and uncertainty.
- Judge the judgesCompare human and AI ratings and investigate disagreement.
- Build meaningful evalsVary inputs, establish expectations, and challenge your answer keys.
- Go beyond the averageTest privacy, safety, and failure conditions that a good mean cannot excuse.
- Decide with evidenceCompare builds and explain the limits of your release confidence.