AI engineering glossary · Transformers LLM architecture
What is top-p sampling?
Top-p sampling (also called Nucleus sampling): A sampling method that sorts tokens by probability and keeps the smallest set whose probabilities add up to p, then samples from that set. The number of candidates adapts: few when the model is confident, more when it is uncertain.
Top-p sampling: an example
With p = 0.9, if 'Delhi' has 0.7 and 'Mumbai' 0.25, only those two are kept because 0.95 exceeds 0.9.
Related terms
More Transformers LLM architecture terms
- Transformer
- Transformer block
- Transformer MLP block
- Transformer-XL
- Typical sampling
- Universal Transformer
- YaRN
- Activation patching
Where you learn it
Top-p sampling is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.