AI engineering glossary · Deep learning core
What is stochastic depth?
Stochastic depth: A training technique for very deep residual networks that randomly skips entire residual blocks on each step, so the network is effectively shallower while training and full depth at test time. It reduces training time and overfitting.
Stochastic depth: an example
In a 100-block network, roughly 20 blocks might be bypassed on a given batch.
Related terms
More Deep learning core terms
- Stochastic weight averaging
- Straight-through estimator
- SwiGLU
- Symmetry breaking
- Tanh
- Teacher forcing
- Training loop
- Universal approximation theorem
Where you learn it
Stochastic depth is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.