AI engineering glossary · Deep learning core
What is batch renormalisation?
Batch renormalisation: An extension of batch normalisation that corrects the mini-batch statistics using running averages of the mean and variance, so training-time and test-time behaviour match more closely. It helps when mini-batches are very small or not evenly mixed.
Batch renormalisation: an example
With batches of only 4 examples, it keeps outputs consistent with what the model will see at inference.
Related terms
More Deep learning core terms
- Batch size
- bfloat16
- Bias term
- Bidirectional RNN
- Binary cross-entropy
- Bottleneck
- Catastrophic forgetting
- Cell state
Where you learn it
Batch renormalisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.