AI engineering glossary · Calculus optimisation
What is convergence?
Convergence: The point at which an iterative algorithm settles, so that further steps no longer change the answer or objective meaningfully. Practitioners watch for the loss to flatten or the gradient norm to become tiny.
Convergence: an example
Training loss falling 2.3, 1.1, 0.62, 0.55, 0.549, 0.549 suggests convergence after the fifth step.
Related terms
More Calculus optimisation terms
- Convergence rate
- Convex function
- Convex optimisation
- Convex set
- Coordinate descent
- Cosine annealing
- Derivative
- Derivative-free optimisation
Where you learn it
Convergence is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.