AI engineering glossary · Calculus optimisation
What is BFGS?
BFGS: The best-known quasi-Newton algorithm, named after Broyden, Fletcher, Goldfarb and Shanno. It maintains a dense approximate inverse Hessian updated with rank-two corrections, and is typically paired with a line search. It suits problems with up to a few thousand variables.
BFGS: an example
scipy.optimize.minimize(f, x0, method="BFGS") runs it on a smooth function of a few dozen parameters.
Related terms
More Calculus optimisation terms
- Chain rule
- Computational graph
- Condition number
- Conjugate gradient method
- Constrained optimisation
- Constraint
- Continuity
- Convergence
Where you learn it
BFGS is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.