Section 069 · Chapter 9, Generated Code Changes the Job
Validation Is the Hard Part of AI-Generated Code
AI makes code generation cheap. It does not make the cost of proving that code safe, correct, and maintainable cheap.
What to do
- Use superlinear growth and the O(n^2) pairwise surface as planning heuristics for interaction risk, not literal forecasts.
- Use dependency analysis, contract checks, risk scoring, mutation testing, property-based tests, historical defect replay, and production trace replay to keep validation efficient as AI-generated code volume rises.
- Define runnable checks that exercise generated code and validation hard part generated code.
Evidence to preserve
- Preserve the inputs, versions, configurations, raw outcomes, and results for generated code, validation hard part generated code needed to reproduce work on Validation Is the Hard Part of AI-Generated Code.
- Report results for generated code, validation hard part generated code by relevant slice, separate blocker failures from averages, state uncertainty and blind spots, and connect the result to a release decision.
Expert note
Estimate validation effort by interaction graph, not lines of code. Use superlinear growth and the O(n^2) pairwise surface as planning heuristics for interaction risk, not literal forecasts. Use dependency analysis, contract checks, risk scoring, mutation testing, property-based tests, historical defect replay, and production trace replay to keep validation efficient as AI-generated code volume rises.
Continue the conversation
Apply this to your context.
Save your product context once, then open a focused conversation that combines it with this concept.
Cite this page
Jason Arbon. "Validation Is the Hard Part of AI-Generated Code." Testing AI Knowledge Edition, section 69.
https://jarbon.ai/testing-ai/knowledge/ch069-validation-hard-part-generated-code.html