AI engineering glossary · Inference serving optimisation
What is unstructured pruning?
Unstructured pruning: Setting individual weights to zero wherever they are least important, without regard to position. It can reach high sparsity with little accuracy loss, but ordinary GPUs cannot exploit scattered zeros, so speedups need special hardware or kernels.
Unstructured pruning: an example
Zeroing 50 percent of weights by magnitude halves the non-zero count, yet dense matrix multiplication takes the same time.
Related terms
More Inference serving optimisation terms
- Verification step
- vLLM
- Warp
- Weight-only quantisation
- Zero-point
- Acceptance rate
- Activation quantisation
- All-reduce
Where you learn it
Unstructured pruning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.