AI engineering glossary · ML fundamentals
What is regularisation?
Regularisation (also called Regularization): Any technique that deliberately restricts or penalises a model's complexity so it fits noise less and generalises better. Typical methods add a penalty on large weights to the loss, stop training early or drop parts of the network. It trades a little training accuracy for better performance on new data.
Regularisation: an example
Adding a penalty of 0.01 times the sum of squared weights to the loss keeps a regression model from using extreme coefficients.
Related terms
More ML fundamentals terms
- Representation learning
- Reproducibility
- Selection bias
- Self-supervised learning
- Semi-supervised learning
- Shortcut learning
- Spurious correlation
- Stratified k-fold
Where you learn it
Regularisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.