AI engineering glossary · Deep learning core
What is ELU?
ELU (also called Exponential linear unit): The exponential linear unit, an activation that behaves like the identity for positive inputs and follows a smooth exponential curve, a*(e^x - 1), for negative ones. Its outputs can be negative, pushing average activations closer to zero, which can speed learning.
ELU: an example
ELU(-1) with a = 1 is e^(-1) - 1, about -0.63; ELU(2) = 2.
Related terms
More Deep learning core terms
- Embedding layer
- Epoch
- Evaluation mode
- Exploding gradient
- Exponential moving average of weights
- Fan-in and fan-out
- Feedforward neural network
- Flat minima
Where you learn it
ELU is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.