AI engineering glossary · Deep learning core
What is SELU?
SELU (also called Scaled exponential linear unit): The scaled exponential linear unit, an activation with fixed constants chosen so that, in a suitably initialised feedforward network, layer outputs tend to keep zero mean and unit variance without explicit normalisation layers. It is called self-normalising.
SELU: an example
A deep MLP using SELU with LeCun-normal initialisation can keep stable activations across many layers.
Related terms
More Deep learning core terms
- SiLU
- Skip connection
- Softmax
- Softplus
- Sparse categorical cross-entropy
- Spectral normalisation
- Stochastic depth
- Stochastic weight averaging
Where you learn it
SELU is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.