AI engineering glossary · Classical ML algorithms
What is feature importance?
Feature importance: A score saying how much each input contributes to a model's predictions. For trees it is commonly the total impurity reduction a feature brings across all splits, averaged over the forest. It guides feature selection and debugging but can mislead with correlated or high-cardinality features.
Feature importance: an example
A forest on loan data reports importance 0.35 for credit_score and 0.02 for city, hinting that credit_score drives most decisions.
Related terms
More Classical ML algorithms terms
- Gaussian mixture model
- Gaussian naive Bayes
- Gaussian process
- Gini impurity
- Gradient boosting
- HDBSCAN
- Hierarchical clustering
- Hinge loss
Where you learn it
Feature importance is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.