AI engineering glossary · Calculus optimisation
What is L-BFGS?
L-BFGS (also called Limited-memory BFGS): A limited-memory version of BFGS that stores only the last handful of parameter and gradient changes, rather than a full matrix, to implicitly represent curvature. It scales to many variables and is used for classical models like logistic regression.
L-BFGS: an example
scikit-learn's LogisticRegression uses the "lbfgs" solver by default, keeping perhaps 10 recent updates.
Related terms
More Calculus optimisation terms
- L-smoothness
- Lagrange multiplier
- Lagrangian
- Learning rate
- Learning rate schedule
- Learning rate warmup
- Level set
- Levenberg-Marquardt algorithm
Where you learn it
L-BFGS is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.