AI engineering glossary · Deep learning core
What is learning rate finder?
Learning rate finder (also called LR range test): A quick experiment that trains for a short time while increasing the learning rate exponentially and records the loss. A good rate is picked from the region where loss falls fastest, before it blows up.
Learning rate finder: an example
Sweeping from 1e-7 to 1 over 100 steps and seeing steepest drop near 3e-3 suggests trying about 1e-3.
Related terms
More Deep learning core terms
- LeCun initialisation
- Linear scaling rule
- Lion optimiser
- Log-sum-exp trick
- Logits
- Loss scaling
- Lottery ticket hypothesis
- LSTM
Where you learn it
Learning rate finder is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.