AI engineering glossary · Deep learning core
What is cyclical learning rate?
Cyclical learning rate (also called Triangular schedule): A policy that swings the learning rate between a lower and an upper bound repeatedly instead of only decreasing it. It can reduce tuning effort and help training cross flat regions or saddle points.
Cyclical learning rate: an example
The rate might rise from 0.001 to 0.01 and back over every 2,000 iterations.
Related terms
More Deep learning core terms
Where you learn it
Cyclical learning rate is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.