AI engineering glossary · Evals guardrails AI security
What is deepEval?
DeepEval: An open-source Python framework for evaluating language model applications with unit-test style checks and ready-made metrics, such as faithfulness, answer relevancy and hallucination. It plugs into test runners like pytest so evaluation can run alongside ordinary code tests.
DeepEval: an example
An engineer adds a test that fails the build if a RAG answer's faithfulness score falls below 0.8.
Related terms
More Evals guardrails AI security terms
- Defence in depth
- Denial of wallet
- Direct prompt injection
- Egress filtering
- Elo rating
- Encoding attack
- Eval set
- Evaluation harness
Where you learn it
DeepEval is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.