AI engineering glossary · Calculus optimisation
What is Quasi-Newton method?
Quasi-Newton method: A family of optimisers that build up an approximation of the inverse Hessian from successive gradient differences instead of computing second derivatives directly. They keep much of Newton's fast convergence at a much lower cost per step.
Quasi-Newton method: an example
BFGS updates its curvature estimate after every step using only the change in the parameters and the change in the gradient.
Related terms
More Calculus optimisation terms
- Quotient rule
- Rate of change
- Reverse-mode automatic differentiation
- Riemann sum
- RMSProp
- Saddle point
- Secant line
- Second-order approximation
Where you learn it
Quasi-Newton method is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.