AI engineering glossary · Transformers LLM architecture
What is length penalty?
Length penalty: An adjustment used in beam search that changes how sequence scores depend on length. Because summed log-probabilities always fall as text grows, unadjusted search favours very short outputs; dividing by a power of the length compensates.
Length penalty: an example
score = sum(log p) / length^alpha, with alpha = 0.6, stops a 5-token answer from beating a better 20-token one.
Related terms
More Transformers LLM architecture terms
- Linear attention
- Llama
- LM head
- Load-balancing loss
- Log probabilities
- Logit lens
- Long-context model
- Longformer
Where you learn it
Length penalty is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.