Section 009 · Chapter 2, From Tests to Release Evidence
Risk-Based Sampling
Testing effort should follow risk. High-impact failures deserve more samples, stricter gates, and deeper review.
risk-based samplingrisk based sampling
What to do
- Test the single-signal cases.
- Test sensor disagreement.
- Test the user profile that points the wrong way.
- Test the judge that is confident but wrong.
- Test what happens when retrieval returns one authoritative-looking but outdated document.
Evidence to preserve
- Preserve the inputs, versions, configurations, raw outcomes, and results for risk-based sampling, risk based sampling needed to reproduce work on Risk-Based Sampling.
- Report results for risk-based sampling, risk based sampling by relevant slice, separate blocker failures from averages, state uncertainty and blind spots, and connect the result to a release decision.
Expert note
Expert risk sampling combines likelihood, severity, detectability, reversibility, and exposure. A rare failure with irreversible harm may deserve more testing than a frequent cosmetic issue.
Continue the conversation
Apply this to your context.
Save your product context once, then open a focused conversation that combines it with this concept.
Cite this page
Jason Arbon. "Risk-Based Sampling." Testing AI Knowledge Edition, section 9.
https://jarbon.ai/testing-ai/knowledge/ch009-risk-based-sampling.html