AI engineering glossary · Evals guardrails AI security
What is HELM?
HELM (also called Holistic Evaluation of Language Models): Holistic Evaluation of Language Models, a Stanford benchmark and framework that evaluates models across many scenarios and several metrics at once, including accuracy, calibration, robustness, fairness, bias, toxicity and efficiency, to give a broader picture than one accuracy number.
HELM: an example
Rather than ranking models on accuracy alone, a HELM-style report also shows how each model does on toxicity and robustness to typos.
Related terms
More Evals guardrails AI security terms
- HumanEval
- Improper output handling
- Input guardrail
- Intrinsic hallucination
- ISO/IEC 42001
- Jailbreak
- Judge calibration
- Least privilege
Where you learn it
HELM is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.