AI engineering glossary · Calculus optimisation
What is linear approximation?
Linear approximation (also called First-order approximation): Estimating a function near a point by its tangent line or plane: f(x) is approximately f(a) + f'(a)(x - a). It is accurate for small steps and is the idea behind first-order optimisers, which trust the gradient only locally.
Linear approximation: an example
sqrt(4.1) is approximately 2 + (1/4)(0.1) = 2.025; the exact value is about 2.0248.
Related terms
More Calculus optimisation terms
- Linear programming
- Lipschitz continuity
- Local minimum
- Loss landscape
- Mini-batch gradient descent
- Momentum
- Multivariable chain rule
- Natural logarithm
Where you learn it
Linear approximation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.