AI engineering glossary · Deep learning core
What is backward pass?
Backward pass (also called Backward propagation step): The half of a training step in which gradients of the loss are computed for all parameters, moving from the output layer toward the input. It typically costs about as much as, or somewhat more than, the forward pass because it reuses stored activations.
Backward pass: an example
After computing the loss on a batch, calling loss.backward() in PyTorch runs this step.
Related terms
More Deep learning core terms
- Batch normalisation
- Batch renormalisation
- Batch size
- bfloat16
- Bias term
- Bidirectional RNN
- Binary cross-entropy
- Bottleneck
Where you learn it
Backward pass is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.