AI engineering glossary · Deep learning core
What is dropout?
Dropout: A regularisation technique that randomly switches off a fraction of neurons during each training step, so the network cannot rely on any single unit. At test time all neurons are used, with values scaled to compensate.
Dropout: an example
With a rate of 0.5, about half of a 100-unit layer outputs 0 on each step.
Related terms
More Deep learning core terms
- Dying ReLU
- ELU
- Embedding layer
- Epoch
- Evaluation mode
- Exploding gradient
- Exponential moving average of weights
- Fan-in and fan-out
Where you learn it
Dropout is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.