AI engineering glossary · LLM training alignment
What is IPO (identity preference optimisation)?
Identity preference optimisation (also called IPO): A variant of direct preference optimisation that replaces the sigmoid-based loss with a squared-error loss on the log-ratio margin, designed to reduce overfitting when preferences are nearly deterministic. It regularises the gap between chosen and rejected log-ratios toward a fixed target.
Identity preference optimisation: an example
When nearly every pair in a dataset has an obvious winner, plain DPO can push the margin to extremes, while IPO holds it near a set value.
Related terms
More LLM training alignment terms
- InstructGPT
- Instruction backtranslation
- Instruction tuning
- Kahneman-Tversky optimisation
- KL penalty
- Knowledge distillation
- LIMA
- Long-context fine-tuning
Where you learn it
Identity preference optimisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.