AI engineering glossary · Calculus optimisation
What is preconditioning?
Preconditioning: Transforming an optimisation problem, usually by rescaling variables or the gradient, so that its curvature is more uniform and the condition number is smaller. Adaptive optimisers such as Adam act as cheap diagonal preconditioners.
Preconditioning: an example
Standardising two features from Rs and square feet to zero mean and unit variance reshapes an elongated loss valley into a rounder bowl.
Related terms
More Calculus optimisation terms
- Product rule
- Projected gradient descent
- Proximal gradient method
- Quasi-Newton method
- Quotient rule
- Rate of change
- Reverse-mode automatic differentiation
- Riemann sum
Where you learn it
Preconditioning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.