AI engineering glossary · Deep learning core
What is one-cycle policy?
One-cycle policy: A schedule that raises the learning rate from a small value to a high peak and then lowers it far below the start, all within one training run. It often allows fast training at large rates, sometimes called super-convergence.
One-cycle policy: an example
A run might rise from 0.004 to 0.04 in the first 30 percent of steps, then fall toward 0.00004.
Related terms
More Deep learning core terms
- Optimiser
- Orthogonal initialisation
- Output layer
- Parametric ReLU
- Recurrent neural network
- ReLU
- RMSNorm
- Saturation
Where you learn it
One-cycle policy is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.