AI engineering glossary · Calculus optimisation
What is learning rate schedule?
Learning rate schedule (also called Learning rate decay, LR scheduler): A rule that changes the learning rate during training, usually starting larger for fast progress and decreasing so the model can settle precisely. Common forms include step decay, exponential decay, cosine annealing and warmup followed by decay.
Learning rate schedule: an example
Start at 0.1 and multiply by 0.1 at epochs 30 and 60, or follow a cosine curve from 0.1 to nearly 0 across 100 epochs.
Related terms
More Calculus optimisation terms
- Learning rate warmup
- Level set
- Levenberg-Marquardt algorithm
- Limit
- Line search
- Linear approximation
- Linear programming
- Lipschitz continuity
Where you learn it
Learning rate schedule is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.