AI engineering glossary · Transformers LLM architecture
What is test-time compute?
Test-time compute (also called Inference-time compute): The computation spent while a model answers, rather than during training. It can be raised by generating longer reasoning, sampling multiple answers and selecting the best, or searching. More of it can lift accuracy on hard problems without a bigger model.
Test-time compute: an example
Sampling 16 solutions to a maths question and taking the majority answer uses 16 times the inference compute of a single answer.
Related terms
More Transformers LLM architecture terms
- Top-k expert routing
- Top-k sampling
- Top-p sampling
- Transformer
- Transformer block
- Transformer MLP block
- Transformer-XL
- Typical sampling
Where you learn it
Test-time compute is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.