AI engineering glossary · Transformers LLM architecture
What is temperature?
Temperature (also called Sampling temperature): A setting that divides the logits by a number before softmax, controlling how spread out the next-token probabilities are. Below 1 makes the model more confident and predictable, above 1 flattens the distribution and adds variety, and near 0 approaches greedy decoding.
Temperature: an example
At temperature 0.2 the top token might get 95 percent probability; at 1.5 the same token might get 40 percent.
Temperature in Python
A short example you can run (Python 3.11, NumPy). The output below is what it really prints.
import numpy as np
logits = np.array([2.0, 1.0, 0.5])
for t in (0.2, 1.0, 2.0):
p = np.exp(logits / t) / np.exp(logits / t).sum()
print(f"T={t}: {np.round(p, 3)}")Output
T=0.2: [0.993 0.007 0.001] T=1.0: [0.629 0.231 0.14 ] T=2.0: [0.481 0.292 0.227]
Related terms
More Transformers LLM architecture terms
- Test-time compute
- Top-k expert routing
- Top-k sampling
- Top-p sampling
- Transformer
- Transformer block
- Transformer MLP block
- Transformer-XL
Where you learn it
Temperature is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.