AI engineering glossary · Calculus optimisation
What is saddle point?
Saddle point: A stationary point that curves upward in some directions and downward in others, so it is neither a minimum nor a maximum. In high-dimensional non-convex problems such as neural network training, these are far more common than poor local minima.
Saddle point: an example
f(x, y) = x^2 - y^2 has zero gradient at the origin, yet moving along x raises f while moving along y lowers it.
Related terms
More Calculus optimisation terms
- Secant line
- Second-order approximation
- Second-order condition
- Sharpness-aware minimisation
- Simulated annealing
- Stationary point
- Step decay
- Stochastic gradient descent
Where you learn it
Saddle point is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.