AI engineering glossary · LLM training alignment
What is emergent abilities?
Emergent abilities: Abilities that seem absent in smaller models and appear at larger scale. Whether they are truly sudden or an artefact of the metric used is debated, since smoother metrics often show gradual improvement.
Emergent abilities: an example
Multi-digit arithmetic accuracy stays near zero for small models and then rises in larger ones on an exact-match metric.
Related terms
More LLM training alignment terms
- Evol-Instruct
- Exposure bias
- FLAN
- FLOPs
- Foundation model
- FP8 training
- Full fine-tuning
- Fully sharded data parallel
Where you learn it
Emergent abilities is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.