AI engineering glossary · Evaluation metrics
What is proper scoring rule?
Proper scoring rule: A scoring function for probability forecasts that is best achieved, in expectation, by reporting one's true beliefs. It cannot be gamed by hedging. Log loss and the Brier score are proper, whereas plain accuracy applied to thresholded predictions is not.
Proper scoring rule: an example
Under the Brier score, if you truly believe rain has a 70% chance, stating 70% earns a better expected score than exaggerating to 100%.
Related terms
More Evaluation metrics terms
- R-precision
- Recall
- Recall at k
- Reciprocal rank
- Relevance judgments
- Reliability diagram
- Residual plot
- ROC curve
Where you learn it
Proper scoring rule is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.