AI engineering glossary · Evals guardrails AI security
What is evaluation harness?
Evaluation harness (also called eval harness): Software that runs a model or application over a set of tasks in a repeatable way, handling prompt formatting, model calls, scoring and result reporting. A shared harness makes results reproducible and comparable between models, and lets a team wire evaluations into automated pipelines.
Evaluation harness: an example
A team runs one command that sends 1,000 test prompts to three candidate models and prints accuracy and cost per model.
Related terms
More Evals guardrails AI security terms
- Evasion attack
- Excessive agency
- Extrinsic hallucination
- Fast Gradient Sign Method
- G-Eval
- Garak
- Greedy Coordinate Gradient
- Groundedness
Where you learn it
Evaluation harness is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.