AI engineering glossary · Deep learning core
What is AMP (mixed precision training)?
Mixed precision training (also called AMP, FP16 training): A method that runs most calculations in 16-bit floating point while keeping a 32-bit copy of the weights for updates. It cuts memory use and speeds up GPUs that have hardware for half precision, with little effect on accuracy when done right.
Mixed precision training: an example
Using autocast in PyTorch with a Colab T4 GPU can noticeably shorten epochs versus pure 32-bit.
Related terms
More Deep learning core terms
- Model parallelism
- Multi-task learning
- Multilayer perceptron
- Negative log-likelihood
- Neural network pruning
- Neural tangent kernel
- Neuron
- One-cycle policy
Where you learn it
Mixed precision training is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.