AI engineering glossary · Calculus optimisation
What is armijo condition?
Armijo condition (also called Sufficient decrease condition): A sufficient-decrease test for step size: a step t along direction d is accepted only if f(x + t*d) <= f(x) + c * t * (gradient . d), with c a small constant such as 1e-4. It ensures each step lowers the objective by an amount proportional to the step and slope.
Armijo condition: an example
If f(x) = 10, gradient . d = -4, t = 0.5 and c = 0.0001, the new value must be at most 10 - 0.0002 = 9.9998.
Related terms
More Calculus optimisation terms
- Automatic differentiation
- Backtracking line search
- Batch gradient descent
- Bayesian optimisation
- BFGS
- Chain rule
- Computational graph
- Condition number
Where you learn it
Armijo condition is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.