AI engineering glossary · Evals guardrails AI security
What is reference-based evaluation?
Reference-based evaluation: Scoring a model output by comparing it with one or more known correct answers, using exact match, overlap measures or a judge that sees the reference. It works well when tasks have a definite right answer, such as extraction or classification, and is weaker for open-ended writing.
Reference-based evaluation: an example
A bot extracts the invoice total; the output '12,450' is compared with the labelled value '12450.00' after normalising the number format.
Related terms
More Evals guardrails AI security terms
- Reference-free evaluation
- Refusal rate
- Role-play jailbreak
- Rubric-based grading
- Safety evaluation
- Sandboxing
- Schema validation
- Self-preference bias
Where you learn it
Reference-based evaluation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.