AI engineering glossary · Inference serving optimisation
What is sparsity?
Sparsity: The fraction of values in a model or tensor that are exactly zero. High sparsity can save memory and compute if hardware or software skips the zeros, which is why it matters for pruning and for mixture-of-experts style designs.
Sparsity: an example
A weight matrix with 800 zeros among 1,000 entries has 80 percent sparsity.
Related terms
More Inference serving optimisation terms
- Speculative decoding
- Spot instance
- Static batching
- Streaming multiprocessor
- Structured pruning
- Symmetric quantisation
- Tail latency
- Target model
Where you learn it
Sparsity is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.