AI engineering glossary · Calculus optimisation
What is backtracking line search?
Backtracking line search: A line search that starts with a large trial step and shrinks it by a constant factor until a sufficient-decrease condition is met. It is cheap, simple and needs only function evaluations.
Backtracking line search: an example
Begin at step 1, halve repeatedly (1, 0.5, 0.25) until the Armijo condition holds.
Related terms
More Calculus optimisation terms
- Batch gradient descent
- Bayesian optimisation
- BFGS
- Chain rule
- Computational graph
- Condition number
- Conjugate gradient method
- Constrained optimisation
Where you learn it
Backtracking line search is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.