AI engineering glossary · Evals guardrails AI security
What is automated red teaming?
Automated red teaming: Using software, often another language model, to generate and refine attack prompts at scale against a target system, instead of relying only on human testers. It finds more varied weaknesses cheaply and can run continuously as models change.
Automated red teaming: an example
An attacker model rewrites a blocked request in dozens of new ways until one gets past the target's safeguards, and the successes are saved as tests.
Related terms
More Evals guardrails AI security terms
- Backdoor attack
- Benchmark saturation
- Canary token
- Chatbot Arena
- Citation checking
- Constitutional classifiers
- Content moderation
- Crescendo attack
Where you learn it
Automated red teaming is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.