AI engineering glossary · Calculus optimisation
What is total derivative?
Total derivative: The derivative of a function with respect to a variable when all its inputs may themselves depend on that variable. It adds up the direct effect and every indirect effect flowing through the other inputs.
Total derivative: an example
If f(x, y) = x*y with x = t and y = t^2, then f = t^3 and df/dt = 3t^2, which equals y*1 + x*2t = t^2 + 2t^2.
Related terms
More Calculus optimisation terms
- Trust region method
- Unconstrained optimisation
- Variance reduction
- Vector-Jacobian product
- Warm restarts
- AdaGrad
- Adam
- AdamW
Where you learn it
Total derivative is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.