AI engineering glossary · Inference serving optimisation
What is QAT (quantisation-aware training)?
Quantisation-aware training (also called QAT): Training or fine-tuning a model while simulating low-precision arithmetic in the forward pass, so the weights learn to tolerate rounding error. It typically preserves more accuracy at very low bit widths than converting afterwards, but it costs extra training compute.
Quantisation-aware training: an example
A speech model is fine-tuned for a few epochs with fake 4-bit rounding inserted, then deployed on a phone chip that only supports integer maths.
Related terms
- Post-training quantisation
- Fake quantisation
- Straight-through estimator
- Quantisation
More Inference serving optimisation terms
- Queueing delay
- RadixAttention
- Request scheduler
- Reserved capacity
- Roofline model
- Scale factor
- Semantic caching
- Semi-structured sparsity
Where you learn it
Quantisation-aware training is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.