AI engineering glossary · Calculus optimisation
What is batch gradient descent?
Batch gradient descent (also called Full-batch gradient descent): A form of gradient descent that computes the gradient of the loss over the entire training set before every parameter update. Each step is accurate and smooth but expensive, and it becomes impractical when data does not fit in memory.
Batch gradient descent: an example
With 1 million training rows, every single update needs a pass over all 1 million before weights change once.
Related terms
More Calculus optimisation terms
- Bayesian optimisation
- BFGS
- Chain rule
- Computational graph
- Condition number
- Conjugate gradient method
- Constrained optimisation
- Constraint
Where you learn it
Batch gradient descent is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.