AI engineering glossary · LLM training alignment
What is GRPO (group relative policy optimisation)?
Group relative policy optimisation (also called GRPO): A reinforcement learning method that samples a group of answers for each prompt and computes each answer's advantage relative to the group's mean and spread, so no separate value network is needed. It was popularised for training reasoning models with verifiable rewards.
Group relative policy optimisation: an example
For one maths question, 8 answers are sampled; those scoring above the group's average reward get positive advantage and are reinforced.
Related terms
More LLM training alignment terms
- Helpful, honest and harmless
- Hugging Face TRL
- Identity preference optimisation
- InstructGPT
- Instruction backtranslation
- Instruction tuning
- Kahneman-Tversky optimisation
- KL penalty
Where you learn it
Group relative policy optimisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.