AI engineering glossary · Classical ML algorithms
What is multicollinearity?
Multicollinearity: A situation in which two or more input features are strongly correlated with each other. The model cannot tell their effects apart, so fitted weights become unstable and swing widely with small data changes, even though overall predictions may stay fine. Dropping or combining features, or using penalised fitting, helps.
Multicollinearity: an example
Including both height in centimetres and height in inches as features makes the weights on each meaningless individually, since one is an exact multiple of the other.
Related terms
More Classical ML algorithms terms
- Multinomial naive Bayes
- Multiple linear regression
- Naive Bayes
- Non-negative matrix factorisation
- Normal equation
- Odds ratio
- One-class SVM
- One-vs-one
Where you learn it
Multicollinearity is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.