AI engineering glossary · Evals guardrails AI security
What is safety evaluation?
Safety evaluation (also called safety eval): Systematic testing of a model or application for harmful behaviour, including dangerous content, bias, privacy leaks and susceptibility to attacks, using benchmarks, red teaming and human review. Results guide release decisions and mitigations.
Safety evaluation: an example
Before launch, a team runs 2,000 safety prompts across categories and reports refusal and harmful-answer rates for each.
Related terms
More Evals guardrails AI security terms
- Sandboxing
- Schema validation
- Self-preference bias
- Sensitive information disclosure
- Sleeper agents
- Spotlighting
- STRIDE threat model
- Structured output validation
Where you learn it
Safety evaluation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.