AI engineering glossary · Calculus optimisation
What is subgradient?
Subgradient: A generalisation of the gradient for convex functions that are not differentiable everywhere: the slope of any line that touches the graph at the point and stays beneath it. At a kink there are many valid choices.
Subgradient: an example
For f(x) = |x| at x = 0, any value between -1 and 1 is a subgradient; frameworks typically use 0 for ReLU at 0.
Related terms
More Calculus optimisation terms
- Subgradient method
- Sum rule
- Symbolic differentiation
- Tangent line
- Taylor series
- Total derivative
- Trust region method
- Unconstrained optimisation
Where you learn it
Subgradient is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.