AI engineering glossary · Calculus optimisation
What is loss landscape?
Loss landscape (also called Loss surface): The surface formed by plotting a model's loss against its parameters. Its hills, valleys, plateaus and saddles determine how easy training is. It is high-dimensional, so it is studied through slices, contours and curvature statistics.
Loss landscape: an example
Slicing the loss of a small network along two random directions through the trained weights yields a 3D surface with a broad basin.
Related terms
More Calculus optimisation terms
- Mini-batch gradient descent
- Momentum
- Multivariable chain rule
- Natural logarithm
- Nesterov momentum
- Newton's method
- Numerical differentiation
- Objective function
Where you learn it
Loss landscape is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.