AI engineering glossary · Calculus optimisation
What is trust region method?
Trust region method: An optimisation approach that builds a simple model, usually quadratic, of the objective and trusts it only within a bounded radius of the current point. The radius grows or shrinks depending on how well predictions match reality.
Trust region method: an example
If the model predicts a loss drop of 2.0 but the true drop is 1.9, the radius may be enlarged; if the true drop is negative, the step is rejected and the radius shrunk.
Related terms
More Calculus optimisation terms
- Unconstrained optimisation
- Variance reduction
- Vector-Jacobian product
- Warm restarts
- AdaGrad
- Adam
- AdamW
- Antiderivative
Where you learn it
Trust region method is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.