AI engineering glossary · Evals guardrails AI security
What is offline evaluation?
Offline evaluation: Testing a model or prompt on stored data before it reaches users, so results are safe, repeatable and cheap to compare. It catches obvious problems but may not reflect real traffic, which is why it is paired with checks on live behaviour.
Offline evaluation: an example
Before launching a new retrieval setup, engineers score it on 500 saved questions and compare recall against the old setup.
Related terms
More Evals guardrails AI security terms
- Online evaluation
- Output guardrail
- Over-refusal
- Overreliance
- OWASP Top 10 for LLM Applications
- Pairwise comparison
- Payload splitting
- Perplexity filter
Where you learn it
Offline evaluation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.