AI engineering glossary · Inference serving optimisation
What is medusa decoding?
Medusa decoding: A speculative approach that adds several extra prediction heads to the main model, each guessing a token further ahead, so no separate draft model is needed. Candidate continuations are checked together using tree attention.
Medusa decoding: an example
Four extra heads propose the tokens at positions +1 to +4, and the verified ones are kept.
Related terms
More Inference serving optimisation terms
Where you learn it
Medusa decoding is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.