Section 105 · Chapter 13, AI Security and Guardrails
Guardrails for AI Systems
Guardrails are the code, policy, permissions, human review, and telemetry around a model that limit what bad outputs can do.
monitoringhuman reviewguardrailguardrails
What to do
- Test bypasses, race conditions, stale state, confusing warnings, over-trusted operators, malformed inputs, fast repeated actions, and cases where the system says something vague like "minor issue" when the correct behavior is to stop hard.
- Do not rely on one guardrail.
- Use defense in depth: product boundaries, model instructions, retrieval filtering, tool permissions, output checks, human approval, sandboxing, monitoring, and rollback.
Evidence to preserve
- Preserve the inputs, versions, configurations, raw outcomes, and results for monitoring, human review, guardrail, guardrails needed to reproduce work on Guardrails for AI Systems.
- Report results for monitoring, human review, guardrail, guardrails by relevant slice, separate blocker failures from averages, state uncertainty and blind spots, and connect the result to a release decision.
Expert note
At scale, guardrail testing is control-system testing. Each control needs an owner, a purpose, a threat model, an allowed behavior set, a blocked behavior set, a fallback, a log schema, and a way to detect drift.
Continue the conversation
Apply this to your context.
Save your product context once, then open a focused conversation that combines it with this concept.
Cite this page
Jason Arbon. "Guardrails for AI Systems." Testing AI Knowledge Edition, section 105.
https://jarbon.ai/testing-ai/knowledge/ch105-guardrails.html