AI engineering glossary · Inference serving optimisation
What is acceptance rate?
Acceptance rate: The fraction of draft tokens that the large model agrees with during speculative decoding. A higher rate means more tokens per large-model step and greater speed-up; a low rate wastes the draft's work and may even slow generation.
Acceptance rate: an example
If the draft proposes 5 tokens per round and on average 3.5 are accepted, the acceptance rate is 70 percent.
Related terms
- Speculative decoding
- Draft model
- Verification step
- Token throughput
More Inference serving optimisation terms
- Activation quantisation
- All-reduce
- Arithmetic intensity
- Asymmetric quantisation
- Autoscaling
- AWQ
- Batch inference
- Block table
Where you learn it
Acceptance rate is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.