AI engineering glossary · Evals guardrails AI security
What is backdoor attack?
Backdoor attack (also called trojan attack): A poisoning attack that makes a model behave normally except when a secret trigger, such as a rare phrase or pixel pattern, appears in the input, at which point it produces attacker-chosen output. It is hard to detect because ordinary tests never include the trigger.
Backdoor attack: an example
A code model writes safe code unless a comment contains a certain rare word, after which it inserts a vulnerability.
Related terms
More Evals guardrails AI security terms
- Benchmark saturation
- Canary token
- Chatbot Arena
- Citation checking
- Constitutional classifiers
- Content moderation
- Crescendo attack
- Cross-tenant leakage
Where you learn it
Backdoor attack is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.