AI engineering glossary · Evals guardrails AI security
What is benchmark saturation?
Benchmark saturation: The situation where leading models score so close to the maximum on a benchmark that it can no longer separate them or track progress. Saturation pushes the community toward harder or newer benchmarks, and it is often worsened when test items leak into training data.
Benchmark saturation: an example
When several models all score above 90 percent on a test, a further two-point gain says little, so researchers move to a harder set.
Related terms
More Evals guardrails AI security terms
- Canary token
- Chatbot Arena
- Citation checking
- Constitutional classifiers
- Content moderation
- Crescendo attack
- Cross-tenant leakage
- Data exfiltration
Where you learn it
Benchmark saturation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.