AI engineering glossary · LLM training alignment
What is RLHF (reinforcement learning from human feedback)?
Reinforcement learning from human feedback (also called RLHF): A method for aligning a model with human preferences: people rank model answers, a reward model is trained to predict those rankings, and the language model is then optimised with reinforcement learning to score highly under the reward model while staying close to its starting behaviour.
Reinforcement learning from human feedback: an example
Annotators compare two answers to a loan query and pick the clearer one; thousands of such picks train a reward model that then steers the chat model.
Related terms
More LLM training alignment terms
- Reinforcement learning with verifiable rewards
- Rejection sampling fine-tuning
- Reward hacking
- Reward model
- Safety fine-tuning
- Scalable oversight
- Scaling laws
- Self-instruct
Where you learn it
Reinforcement learning from human feedback is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.