AI engineering glossary · Classical ML algorithms
What is permutation importance?
Permutation importance: A way to measure how much a model depends on a feature: shuffle that feature's values in held-out data and see how much the score drops. A large drop means the model relies on it. It works with any model and avoids biases of split-based importance.
Permutation importance: an example
Shuffling the 'tenure' column drops accuracy from 0.90 to 0.82, so tenure has an importance of 0.08.
Related terms
- Feature importance
- Random forest
- Feature selection
- Explainability
More Classical ML algorithms terms
- Polynomial kernel
- Polynomial regression
- Principal component
- Principal component analysis
- Quadratic discriminant analysis
- Random forest
- RBF kernel
- Regression tree
Where you learn it
Permutation importance is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.