AI engineering glossary · Calculus optimisation
What is ISTA (proximal gradient method)?
Proximal gradient method (also called ISTA): An algorithm for minimising a smooth function plus a simple non-smooth term, such as an L1 penalty. It alternates a gradient step on the smooth part with a proximal step that handles the other part exactly, for example soft-thresholding.
Proximal gradient method: an example
For Lasso, take a gradient step on the squared error, then shrink each coefficient toward zero by lambda * lr, setting small ones exactly to zero.
Related terms
More Calculus optimisation terms
- Quasi-Newton method
- Quotient rule
- Rate of change
- Reverse-mode automatic differentiation
- Riemann sum
- RMSProp
- Saddle point
- Secant line
Where you learn it
Proximal gradient method is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.