AI engineering glossary · Calculus optimisation
What is convergence rate?
Convergence rate: A measure of how quickly an iterative method approaches its solution, described as sublinear, linear (error shrinks by a constant factor each step) or quadratic (error roughly squares each step). It lets you compare optimisers on theory.
Convergence rate: an example
Gradient descent on a strongly convex smooth function has a linear rate, while Newton's method near the minimum has a quadratic rate, e.g. error 0.1 becoming 0.01 then 0.0001.
Related terms
More Calculus optimisation terms
- Convex function
- Convex optimisation
- Convex set
- Coordinate descent
- Cosine annealing
- Derivative
- Derivative-free optimisation
- Differentiability
Where you learn it
Convergence rate is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.