AI engineering glossary · Calculus optimisation
What is gradient?
Gradient (also called Nabla f, Del operator): The vector of all partial derivatives of a scalar function of several variables. At any point it points in the direction of steepest increase, and its length gives the steepness. Training a model means repeatedly stepping opposite this vector for the loss.
Gradient: an example
For f(x, y) = x^2 + y^2 at (3, 4), the gradient is (6, 8), so moving along (6, 8) raises f fastest and moving along (-6, -8) lowers it fastest.
Related terms
More Calculus optimisation terms
- Gradient accumulation
- Gradient checking
- Gradient clipping
- Gradient descent
- Hessian matrix
- Integral
- Jacobian matrix
- Jacobian-vector product
Where you learn it
Gradient is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.