AI engineering glossary · ML fundamentals
What is hyperparameter tuning?
Hyperparameter tuning (also called Hyperparameter optimisation): The process of searching for the hyperparameter values that give the best validation performance. Methods range from trying a grid of values to random sampling or smarter guided search. Because each trial means training a model, budget and time matter.
Hyperparameter tuning: an example
Trying 20 combinations of tree depth and learning rate, each scored by 5-fold cross-validation, and keeping the best.
Related terms
More ML fundamentals terms
- Hypothesis space
- Independent and identically distributed
- Inductive bias
- Inference
- Inter-annotator agreement
- Irreducible error
- k-fold cross-validation
- L1 regularisation
Where you learn it
Hyperparameter tuning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.