AI engineering glossary · Transformers LLM architecture
What is greedy decoding?
Greedy decoding (also called Argmax decoding): A decoding strategy that picks the single most probable token at every step. It is fast and deterministic, but can produce repetitive text and may miss sequences that are better overall because an early choice was only locally best.
Greedy decoding: an example
If probabilities are 'Delhi' 0.6 and 'Mumbai' 0.3, greedy decoding always outputs 'Delhi'.
Related terms
More Transformers LLM architecture terms
- Grouped-query attention
- Head dimension
- Hidden states
- Induction head
- Instruct model
- KV cache
- Large language model
- Learned positional embedding
Where you learn it
Greedy decoding is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.