Section 102 · Chapter 13, AI Security and Guardrails
Training Data Poisoning and Backdoors
Bad data can teach a model behavior that only appears when the trigger is right.
What to do
- Ask where examples came from, who labeled them, which synthetic generator created them, which documents were indexed, which production traces became training data, which user feedback was trusted, and whether the eval set itself has been contaminated.
- Do not let generated examples silently become truth without review, especially in safety, medical, legal, financial, or policy domains.
- Test whether malicious, stale, low-authority, or cross-tenant documents can enter retrieval and then become answer evidence.
- Test whether thumbs-up/down, support tickets, bug reports, reviews, or user corrections can be gamed into changing the system.
- Build eval cases that vary rare phrases, file names, comments, image artifacts, metadata, domains, usernames, and formatting to see whether behavior changes unexpectedly.
Evidence to preserve
- Preserve the inputs, versions, configurations, raw outcomes, and results for RAG, synthetic data, data poisoning, backdoor needed to reproduce work on Training Data Poisoning and Backdoors.
- Report results for RAG, synthetic data, data poisoning, backdoor by relevant slice, separate blocker failures from averages, state uncertainty and blind spots, and connect the result to a release decision.
Expert note
In production work, use data provenance, anomaly detection, trigger sweeps, canary tokens, source reputation, fine-tune review, RAG document quarantine, feedback-loop rate limits, synthetic-data labeling, and adversarial evals. Backdoor testing should include negative controls: similar inputs without the trigger should not fail, and trigger-like inputs in harmless contexts should not create false alarms.
Continue the conversation
Apply this to your context.
Save your product context once, then open a focused conversation that combines it with this concept.
Cite this page
Jason Arbon. "Training Data Poisoning and Backdoors." Testing AI Knowledge Edition, section 102.
https://jarbon.ai/testing-ai/knowledge/ch102-training-data-poisoning-backdoors.html