AI engineering glossary · Calculus optimisation
What is forward-mode automatic differentiation?
Forward-mode automatic differentiation (also called Forward mode): A variant that propagates derivatives alongside values from inputs toward outputs, computing one directional derivative per pass. It is efficient when a function has few inputs and many outputs, and it computes Jacobian-vector products.
Forward-mode automatic differentiation: an example
A simulation with 3 tunable parameters and 10,000 outputs needs only 3 forward passes to get its full Jacobian.
Related terms
More Calculus optimisation terms
- Function
- Fundamental theorem of calculus
- Gauss-Newton method
- Genetic algorithm
- Global minimum
- Gradient
- Gradient accumulation
- Gradient checking
Where you learn it
Forward-mode automatic differentiation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.