AI engineering glossary · LLM training alignment
What is Bradley-Terry model?
Bradley-Terry model: A statistical model for pairwise comparisons that says the chance item i beats item j is sigmoid(score_i - score_j). Reward models in RLHF are typically trained with this model: the loss is -log sigmoid(r_chosen - r_rejected), which pushes the chosen response's score above the rejected one's.
Bradley-Terry model: an example
If the chosen score is 1.5 and the rejected score is 0.5, the modelled win probability is sigmoid(1.0), about 0.73.
Related terms
More LLM training alignment terms
- Causal language modelling
- Chat template
- Chinchilla scaling
- Chosen and rejected responses
- Clipped surrogate objective
- Common Crawl
- Compute-optimal training
- Constitutional AI
Where you learn it
Bradley-Terry model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.