Anti-Patterns: The Aggregate Score Trap

What to do

1. Do not create so many slices that every result becomes noise.
2. Choose the ones that matter, then ensure they have enough sample size or targeted evidence.
3. Define slice thresholds before the run.
4. Use confidence intervals per slice, risk-weighted reporting, and minimum quality bars for groups where failure has high cost.

Evidence to preserve

• Include protected classes when relevant, regulatory categories, high-value workflows, high-risk actions, and historically weak segments.
• Preserve the inputs, versions, configurations, raw outcomes, and results for RAG, aggregate score, aggregate score trap needed to reproduce work on Anti-Patterns: The Aggregate Score Trap.
• Report results for RAG, aggregate score, aggregate score trap by relevant slice, separate blocker failures from averages, state uncertainty and blind spots, and connect the result to a release decision.