AI engineering glossary · Transformers LLM architecture
What is typical sampling?
Typical sampling (also called Locally typical sampling): A sampling method that keeps tokens whose surprise, the negative log-probability, is close to the expected surprise of the distribution, instead of just the most probable ones. It aims for text that reads like natural human writing, avoiding both dull and bizarre tokens.
Typical sampling: an example
Very probable filler tokens and very improbable oddities are both filtered, leaving tokens with typical information content.
Related terms
More Transformers LLM architecture terms
- Universal Transformer
- YaRN
- Activation patching
- Active parameters
- Additive attention
- ALiBi
- Attention head
- Attention mechanism
Where you learn it
Typical sampling is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.