AI engineering glossary · Inference serving optimisation
What is target model?
Target model: The large, high-quality model whose outputs are the ones actually wanted, in setups where a smaller helper model proposes text first. Its verification decides which proposed tokens are accepted, so final quality equals what it alone would produce.
Target model: an example
A 70B model acts as the target while a 1B model drafts.
Related terms
More Inference serving optimisation terms
- Tensor core
- TensorRT
- TensorRT-LLM
- Time per output token
- Time to first token
- Tokens per second
- torch.compile
- Total cost of ownership
Where you learn it
Target model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.