AI engineering glossary · Inference serving optimisation
What is SLM (small language model)?
Small language model (also called SLM): A language model with relatively few parameters, from tens of millions to a few billion, designed to run cheaply on modest hardware or devices. It is often good enough for narrow tasks and is much cheaper and faster to serve.
Small language model: an example
A 3B-parameter model fine-tuned on Hindi customer emails classifies them on a single small GPU.
Related terms
More Inference serving optimisation terms
- SmoothQuant
- Sparsity
- Speculative decoding
- Spot instance
- Static batching
- Streaming multiprocessor
- Structured pruning
- Symmetric quantisation
Where you learn it
Small language model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.