AI engineering glossary · Inference serving optimisation
What is AWQ?
AWQ (also called Activation-aware Weight Quantisation): Activation-aware Weight Quantisation, a post-training method that looks at activations to find the small share of weights that matter most and protects them by scaling before quantising to 4 bits. It is fast, needs little calibration data and is widely supported by serving engines.
AWQ: an example
An AWQ 4-bit copy of a 7B model is downloaded from a model hub and loaded straight into a serving engine.
Related terms
More Inference serving optimisation terms
- Batch inference
- Block table
- Cache-aware routing
- Calibration dataset
- Chunked prefill
- Cold start
- Compute-bound
- Continuous batching
Where you learn it
AWQ is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.