AI engineering glossary · Calculus optimisation
What is VJP (Vector-Jacobian product)?
Vector-Jacobian product (also called VJP): The product of a row vector with a Jacobian matrix, giving how a weighted combination of outputs changes with each input. Reverse-mode differentiation computes it directly without ever building the full Jacobian, which keeps memory small.
Vector-Jacobian product: an example
In PyTorch, y.backward(gradient=v) computes v^T J for a vector output y, where v holds upstream sensitivities.
Related terms
More Calculus optimisation terms
- Warm restarts
- AdaGrad
- Adam
- AdamW
- Antiderivative
- Armijo condition
- Automatic differentiation
- Backtracking line search
Where you learn it
Vector-Jacobian product is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.