AI engineering glossary · Calculus optimisation
What is l-smoothness?
L-smoothness (also called Lipschitz gradient): A property of a differentiable function whose gradient is Lipschitz continuous with constant L, meaning the slope cannot change faster than L allows. It implies gradient descent with step size 1/L is guaranteed to decrease the objective every iteration.
L-smoothness: an example
f(x) = 2x^2 has gradient 4x, which is 4-Lipschitz, so it is 4-smooth and a step size of 0.25 is safe.
Related terms
More Calculus optimisation terms
- Lagrange multiplier
- Lagrangian
- Learning rate
- Learning rate schedule
- Learning rate warmup
- Level set
- Levenberg-Marquardt algorithm
- Limit
Where you learn it
L-smoothness is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.