AI engineering glossary · Inference serving optimisation
What is verification step?
Verification step: In speculative decoding, the single forward pass of the large model over all proposed draft tokens at once. It computes the large model's own probabilities for each position, letting the system accept the matching prefix and resample the first mismatch.
Verification step: an example
After 4 draft tokens, one forward pass yields 5 sets of probabilities: one per draft position plus a bonus next token.
Related terms
More Inference serving optimisation terms
- vLLM
- Warp
- Weight-only quantisation
- Zero-point
- Acceptance rate
- Activation quantisation
- All-reduce
- Arithmetic intensity
Where you learn it
Verification step is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.