AI engineering glossary · Deep learning core
What is spectral normalisation?
Spectral normalisation: A technique that divides a layer's weight matrix by its largest singular value so the layer cannot stretch inputs by more than a fixed factor. It stabilises training of models such as GAN discriminators.
Spectral normalisation: an example
If a weight matrix has largest singular value 4, every entry is divided by 4 so the layer scales inputs by at most 1.
Related terms
More Deep learning core terms
- Stochastic depth
- Stochastic weight averaging
- Straight-through estimator
- SwiGLU
- Symmetry breaking
- Tanh
- Teacher forcing
- Training loop
Where you learn it
Spectral normalisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.