AI engineering glossary · Deep learning core
What is skip connection?
Skip connection (also called Shortcut connection): A shortcut that adds a layer's input directly to a later layer's output, bypassing the layers in between. It gives gradients a direct path backward, which makes very deep networks trainable, and it is central to residual networks and transformers.
Skip connection: an example
Output = F(x) + x, so if F(x) is near zero the block simply passes x along.
Related terms
- Residual connection
- Vanishing gradient
- Deep neural network
More Deep learning core terms
- Softmax
- Softplus
- Sparse categorical cross-entropy
- Spectral normalisation
- Stochastic depth
- Stochastic weight averaging
- Straight-through estimator
- SwiGLU
Where you learn it
Skip connection is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.