AI engineering glossary · Deep learning core
What is optimiser?
Optimiser (also called Optimizer): The algorithm that uses gradients to decide how to update weights after each step. Options range from plain gradient descent to methods with momentum or per-parameter learning rates, such as Adam.
Optimiser: an example
torch.optim.SGD(model.parameters(), lr=0.01) creates one; Adam is a common alternative.
Related terms
More Deep learning core terms
- Orthogonal initialisation
- Output layer
- Parametric ReLU
- Recurrent neural network
- ReLU
- RMSNorm
- Saturation
- SELU
Where you learn it
Optimiser is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.