AI engineering glossary · Inference serving optimisation
What is block table?
Block table: A lookup structure mapping a request's logical token positions to the physical memory blocks that hold its KV cache. It lets attention kernels find scattered blocks and lets several requests share identical blocks safely.
Block table: an example
Sequence A's table says logical blocks 0, 1, 2 live in physical blocks 7, 3 and 12.
Related terms
More Inference serving optimisation terms
- Cache-aware routing
- Calibration dataset
- Chunked prefill
- Cold start
- Compute-bound
- Continuous batching
- Cost per token
- CPU offloading
Where you learn it
Block table is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.