AI engineering glossary · Calculus optimisation
What is sum rule?
Sum rule: A differentiation rule saying the derivative of a sum of functions is the sum of their derivatives, and constant multiples pass straight through. It lets you differentiate a long expression piece by piece, which is how per-example loss terms are handled.
Sum rule: an example
d/dx of (4x^3 + 2 sin x) = 12x^2 + 2 cos x. A total loss summed over 100 samples has gradient equal to the sum of 100 per-sample gradients.
Related terms
More Calculus optimisation terms
- Symbolic differentiation
- Tangent line
- Taylor series
- Total derivative
- Trust region method
- Unconstrained optimisation
- Variance reduction
- Vector-Jacobian product
Where you learn it
Sum rule is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.