AI engineering glossary · Inference serving optimisation
What is outlier features?
Outlier features (also called Activation outliers): A few hidden dimensions in large language models whose activations are far bigger than the rest. They make naive low-precision quantisation of activations lose accuracy, and much research on quantisation exists to handle them.
Outlier features: an example
One channel with values near 100 while others sit near 1 forces a per-tensor INT8 scale so coarse that ordinary channels lose most of their precision.
Related terms
More Inference serving optimisation terms
- PagedAttention
- Per-channel quantisation
- Per-tensor quantisation
- Percentile latency
- Post-training quantisation
- Preemption
- Prefill-decode disaggregation
- Prefix caching
Where you learn it
Outlier features is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.