Section 059 · Chapter 8, Operating AI: Observability, Relevance, and Economics
Data Contracts for AI Systems
AI systems need explicit contracts for what they receive, produce, cite, log, refuse, and do.
data contractrefusaldata contracts
What to do
- Start with input contracts.
- Define required fields, allowed formats, maximum sizes, language assumptions, privacy classifications, and what happens when data is missing or malformed.
- Define prompt contracts.
- Define tool-call contracts.
- Define output contracts.
Evidence to preserve
- Preserve the inputs, versions, configurations, raw outcomes, and results for data contract, refusal, data contracts needed to reproduce work on Data Contracts for AI Systems.
- Report results for data contract, refusal, data contracts by relevant slice, separate blocker failures from averages, state uncertainty and blind spots, and connect the result to a release decision.
Expert note
In production work, AI data contracts should be machine-validated, versioned, attached to traces, enforced at runtime, and tested with malformed inputs, adversarial prompts, missing fields, tool errors, and policy changes.
Continue the conversation
Apply this to your context.
Save your product context once, then open a focused conversation that combines it with this concept.
Cite this page
Jason Arbon. "Data Contracts for AI Systems." Testing AI Knowledge Edition, section 59.
https://jarbon.ai/testing-ai/knowledge/ch059-data-contracts.html