AI engineering glossary · Evals guardrails AI security
What is many-shot jailbreaking?
Many-shot jailbreaking: A jailbreak that fills a long context window with many fabricated dialogues in which an assistant complies with harmful requests, so the model imitates the pattern in-context and complies with the final real request. It exploits long-context models and grows stronger with more examples.
Many-shot jailbreaking: an example
An attacker pastes hundreds of fake Q-and-A pairs where the 'assistant' answers dangerous questions, then asks a new one.
Related terms
More Evals guardrails AI security terms
- Membership inference attack
- Misinformation risk
- MITRE ATLAS
- MMLU
- Model extraction
- Model inversion
- Model signing
- Model supply chain attack
Where you learn it
Many-shot jailbreaking is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.