AI engineering glossary · ML fundamentals
What is double descent?
Double descent: A pattern in which test error first falls, then rises as a model nears the point where it can just fit the training data exactly, and then falls again as the model grows even larger. It challenges the simple view that more capacity always hurts generalisation.
Double descent: an example
Plotting test error against network size shows a peak at the size where the model barely memorises the data, then improvement beyond it.
Related terms
More ML fundamentals terms
- Early stopping
- Empirical risk minimisation
- Feature
- Feature space
- Generalisation
- Generalisation error
- Generalisation gap
- Generative model
Where you learn it
Double descent is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.