AI engineering glossary · Calculus optimisation
What is finite difference?
Finite difference: An approximation of a derivative computed from function values at nearby points, such as (f(x + h) - f(x)) / h. It needs no formula for the derivative but suffers from rounding and truncation error, so it is mostly used to verify analytic gradients.
Finite difference: an example
For f(x) = x^2 at x = 3 with h = 0.001: (9.006001 - 9) / 0.001 = 6.001, close to the true derivative 6.
Related terms
More Calculus optimisation terms
- First-order condition
- Flat minimum
- Forward-mode automatic differentiation
- Function
- Fundamental theorem of calculus
- Gauss-Newton method
- Genetic algorithm
- Global minimum
Where you learn it
Finite difference is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.