AI engineering glossary · Evaluation metrics
What is mcNemar's test?
McNemar's test: A statistical test for comparing two classifiers evaluated on the same test set. It looks only at the cases where exactly one model is right and asks whether the two disagreement counts differ more than chance would explain. It suits paired binary outcomes.
McNemar's test: an example
Model A is right and B wrong on 30 cases; B is right and A wrong on 12. McNemar's test asks whether 30 versus 12 is too lopsided to be chance.
Related terms
More Evaluation metrics terms
- Mean absolute error
- Mean absolute percentage error
- Mean absolute scaled error
- Mean average precision
- Mean bias error
- Mean reciprocal rank
- Mean squared error
- Mean squared logarithmic error
Where you learn it
McNemar's test is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.