AI engineering glossary · ML fundamentals
What is occam's razor?
Occam's razor (also called Law of parsimony): The principle that among explanations or models that fit the data about equally well, the simpler one should be preferred. In machine learning it motivates regularisation and model selection criteria, since simpler models tend to generalise better and are easier to interpret.
Occam's razor: an example
If a 3-feature model and a 60-feature model both reach about 88% validation accuracy, the 3-feature one is the better choice.
Related terms
More ML fundamentals terms
- Online learning
- Out-of-distribution
- Outlier
- Overfitting
- Overparameterisation
- PAC learning
- Parameter
- Parametric model
Where you learn it
Occam's razor is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.