AI engineering glossary · LLM training alignment
What is KTO (Kahneman-Tversky optimisation)?
Kahneman-Tversky optimisation (also called KTO): An alignment method that uses only a binary signal per response, 'desirable' or 'undesirable', instead of paired comparisons. Its loss is inspired by prospect theory from behavioural economics, treating gains and losses asymmetrically. It suits data where thumbs-up and thumbs-down are easier to collect than rankings.
Kahneman-Tversky optimisation: an example
App feedback with thumbs-up on 40,000 replies and thumbs-down on 15,000 others can be used directly, without matching them into pairs.
Related terms
More LLM training alignment terms
- KL penalty
- Knowledge distillation
- LIMA
- Long-context fine-tuning
- LoRA
- Loss masking
- Loss spike
- Masked language modelling
Where you learn it
Kahneman-Tversky optimisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.