AI engineering glossary · Calculus optimisation
What is projected gradient descent?
Projected gradient descent: A constrained version of gradient descent that takes an ordinary gradient step and then projects the result back to the nearest point in the feasible set. It suits simple constraints such as boxes or norm balls.
Projected gradient descent: an example
To keep weights within [0, 1], take a step, then clip any value below 0 to 0 and above 1 to 1.
Related terms
More Calculus optimisation terms
- Proximal gradient method
- Quasi-Newton method
- Quotient rule
- Rate of change
- Reverse-mode automatic differentiation
- Riemann sum
- RMSProp
- Saddle point
Where you learn it
Projected gradient descent is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.