AI engineering glossary · Evaluation metrics
What is cohen's kappa?
Cohen's kappa (also called kappa statistic): A measure of agreement between two raters, or between a model and the true labels, that corrects for the agreement expected by chance: kappa = (observed agreement - chance agreement) / (1 - chance agreement). A value of 1 is perfect agreement and 0 is chance level. It is useful when classes are imbalanced.
Cohen's kappa: an example
Two doctors label 100 scans and agree on 90, but chance alone would give 70 agreements, so kappa = (0.90 - 0.70) / (1 - 0.70), about 0.67.
Related terms
More Evaluation metrics terms
- Confusion matrix
- Continuous ranked probability score
- Cost matrix
- Cranfield paradigm
- Cumulative gains chart
- Davies-Bouldin index
- Diagnostic odds ratio
- Discounted cumulative gain
Where you learn it
Cohen's kappa is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.