AI engineering glossary · Inference serving optimisation
What is pruning?
Pruning: Removing weights, neurons, attention heads or whole layers that contribute little to a model's output, then usually fine-tuning to recover accuracy. It shrinks the model and, if the removed parts are structured, can speed up inference.
Pruning: an example
Dropping 30 percent of the smallest-magnitude weights in a vision model and fine-tuning for one epoch recovers most of the original accuracy.
Related terms
More Inference serving optimisation terms
- Quantisation
- Quantisation-aware training
- Queueing delay
- RadixAttention
- Request scheduler
- Reserved capacity
- Roofline model
- Scale factor
Where you learn it
Pruning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.