AI engineering glossary · Calculus optimisation
What is step decay?
Step decay: A learning-rate schedule that cuts the rate by a fixed factor after a set number of epochs or steps. It is simple and effective, producing a staircase-shaped curve as training progresses and letting weights settle precisely.
Step decay: an example
Learning rate 0.1 is halved every 10 epochs: 0.1, 0.05, 0.025, 0.0125.
Related terms
More Calculus optimisation terms
- Stochastic gradient descent
- Strong convexity
- Subgradient
- Subgradient method
- Sum rule
- Symbolic differentiation
- Tangent line
- Taylor series
Where you learn it
Step decay is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.