AI engineering glossary · LLM training alignment
What is scalable oversight?
Scalable oversight: Research on how humans can reliably supervise AI systems whose outputs are too complex or numerous to check directly, for example by using AI assistants, task decomposition or debate to help evaluators. It matters as models surpass human ability on some tasks.
Scalable oversight: an example
A reviewer checks a 500-line program by asking an assistant model to point out its riskiest lines.
Related terms
More LLM training alignment terms
- Scaling laws
- Self-instruct
- Self-rewarding language models
- Self-Taught Reasoner
- Sequence packing
- Sequence-level distillation
- Simple preference optimisation
- Soft labels
Where you learn it
Scalable oversight is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.