AI engineering glossary · Evals guardrails AI security
What is model extraction?
Model extraction (also called model stealing): An attack that reconstructs a copy of a proprietary model, or its behaviour, by sending it many queries and training a substitute on the answers. It steals intellectual property and helps attackers craft attacks offline, so defences include rate limits and query monitoring.
Model extraction: an example
A competitor sends a million prompts to a paid API, records the replies and fine-tunes its own model to imitate it.
Related terms
More Evals guardrails AI security terms
- Model inversion
- Model signing
- Model supply chain attack
- Multimodal prompt injection
- NeMo Guardrails
- NIST AI Risk Management Framework
- Offline evaluation
- Online evaluation
Where you learn it
Model extraction is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.