AI engineering glossary · Calculus optimisation
What is unconstrained optimisation?
Unconstrained optimisation: Optimisation in which the variables may take any real values, with no restrictions. Most neural-network training is treated this way, and it is solved by following gradient information to a point where the slope vanishes.
Unconstrained optimisation: an example
Minimising f(w) = (w - 5)^2 over all real w gives w = 5.
Related terms
More Calculus optimisation terms
- Variance reduction
- Vector-Jacobian product
- Warm restarts
- AdaGrad
- Adam
- AdamW
- Antiderivative
- Armijo condition
Where you learn it
Unconstrained optimisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.