AI engineering glossary · Evaluation metrics
What is MCE (maximum calibration error)?
Maximum calibration error (also called MCE): The largest gap between confidence and accuracy over all confidence bins, rather than the average gap. It measures the worst-case miscalibration and is relevant for high-stakes uses where a single badly calibrated confidence range could cause harm.
Maximum calibration error: an example
If bin gaps are 0.02, 0.05 and 0.18, the ECE-style average is small but the maximum calibration error is 0.18.
Related terms
More Evaluation metrics terms
- McNemar's test
- Mean absolute error
- Mean absolute percentage error
- Mean absolute scaled error
- Mean average precision
- Mean bias error
- Mean reciprocal rank
- Mean squared error
Where you learn it
Maximum calibration error is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.