AI engineering glossary · Evaluation metrics
What is weighted kappa?
Weighted kappa: A version of Cohen's kappa for ordered categories in which disagreements are penalised according to how far apart the two labels are, usually linearly or quadratically. Confusing 'mild' with 'severe' counts as worse than confusing 'mild' with 'moderate'. It is common for graded ratings and essay scoring.
Weighted kappa: an example
Two examiners grade essays 1 to 5. Disagreeing by one grade costs little under quadratic weighting, while a 1 versus a 5 costs sixteen times as much.
Related terms
More Evaluation metrics terms
- Word error rate
- Youden's J statistic
- Accuracy
- Accuracy paradox
- Adjusted Rand index
- AUC-ROC
- Average precision
- Balanced accuracy
Where you learn it
Weighted kappa is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.