AI engineering glossary · Calculus optimisation
What is derivative-free optimisation?
Derivative-free optimisation (also called Black-box optimisation, Gradient-free optimisation): Methods that optimise a function using only its values, with no gradient information. They are used when the function is a black box, non-differentiable or noisy, as with hyperparameter tuning or simulation. Nelder-Mead and evolutionary methods are examples.
Derivative-free optimisation: an example
Tuning a learning rate and batch size by trying combinations and measuring validation accuracy needs no gradient of accuracy with respect to those settings.
Related terms
More Calculus optimisation terms
- Differentiability
- Directional derivative
- Duality
- Euler's number
- Exponential function
- Feasible region
- Finite difference
- First-order condition
Where you learn it
Derivative-free optimisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.