AI engineering glossary · Inference serving optimisation
What is draft model?
Draft model: The small, fast model in speculative decoding that guesses upcoming tokens for the large model to verify. It should resemble the large model's behaviour, ideally sharing its tokeniser, so that many guesses are accepted.
Draft model: an example
A 68M-parameter model from the same family drafts tokens for a 7B target.
Related terms
More Inference serving optimisation terms
Where you learn it
Draft model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.