AI engineering glossary · Evaluation metrics
What is pinball loss?
Pinball loss (also called quantile loss): The loss used to train and evaluate quantile forecasts. For quantile level tau it charges tau * (y - q) when the actual value is above the prediction q and (1 - tau) * (q - y) when it is below. It is asymmetric, so low or high quantiles are judged appropriately.
Pinball loss: an example
For the 0.9 quantile, an actual of 100 and prediction of 90 costs 0.9 * 10 = 9, while a prediction of 110 costs only 0.1 * 10 = 1.
Related terms
- Mean absolute error
- Prediction interval coverage
- Continuous ranked probability score
- Mean squared error
More Evaluation metrics terms
- Platt scaling
- Precision at k
- Precision-recall break-even point
- Precision-recall curve
- Precision-recall trade-off
- Prediction interval coverage
- Prevalence
- Proper scoring rule
Where you learn it
Pinball loss is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.