AI engineering glossary · Evals guardrails AI security
What is threat modelling?
Threat modelling: A structured exercise to identify what could go wrong in a system: what assets need protecting, who might attack, how they could do it and which controls would stop them. For AI apps it examines data flows, tools, prompts and models, ideally before the system is built.
Threat modelling: an example
Before launching an agent that reads email, the team lists assets, trust boundaries and injection paths, and decides to block outbound web calls.
Related terms
More Evals guardrails AI security terms
- Tool poisoning
- Tool-call accuracy
- Topical rails
- Toxicity classification
- Training data extraction
- TruthfulQA
- Unbounded consumption
- Vector and embedding weaknesses
Where you learn it
Threat modelling is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.