AI engineering glossary · Calculus optimisation
What is flat minimum?
Flat minimum: A minimum surrounded by a wide region where the loss stays low, meaning small changes in the parameters barely affect it. Such minima are commonly linked to better generalisation than sharp minima, though the link depends on how sharpness is measured.
Flat minimum: an example
Two models reach the same training loss of 0.10; nudging the first one's weights by 1% keeps loss near 0.10, but the second jumps to 0.60, so the first sits in a flatter minimum.
Related terms
More Calculus optimisation terms
- Forward-mode automatic differentiation
- Function
- Fundamental theorem of calculus
- Gauss-Newton method
- Genetic algorithm
- Global minimum
- Gradient
- Gradient accumulation
Where you learn it
Flat minimum is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.