AI engineering glossary · Deep learning core
What is training loop?
Training loop: The repeated procedure that trains a network: fetch a batch, run the forward pass, compute the loss, run the backward pass, update weights with an optimiser and clear gradients. Evaluation on validation data is usually interleaved.
Training loop: an example
In PyTorch, a loop calls optimizer.zero_grad(), loss.backward() and optimizer.step() for every batch.
Related terms
More Deep learning core terms
- Universal approximation theorem
- Validation loss
- Vanishing gradient
- Weight decay
- Weight initialisation
- Weight normalisation
- Weight sharing
- Weight tying
Where you learn it
Training loop is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.