AI engineering glossary · Calculus optimisation
What is second-order approximation?
Second-order approximation: Approximating a function near a point using its value, gradient and Hessian: f(x + d) is roughly f(x) + g.d + 0.5 * d.H.d. The quadratic term captures curvature, letting an optimiser choose better step sizes and directions than the gradient alone.
Second-order approximation: an example
For f(x) = x^4 at x = 1, the approximation is 1 + 4d + 6d^2; for d = 0.1 it gives 1.46 against the true value 1.4641.
Related terms
More Calculus optimisation terms
- Second-order condition
- Sharpness-aware minimisation
- Simulated annealing
- Stationary point
- Step decay
- Stochastic gradient descent
- Strong convexity
- Subgradient
Where you learn it
Second-order approximation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.