AI engineering glossary · LLM training alignment
What is GAE (generalised advantage estimation)?
Generalised advantage estimation (also called GAE): A technique that computes advantages as a weighted sum of temporal-difference errors, with a parameter lambda that trades bias against variance. Setting lambda near 0 relies on the value estimate, while lambda near 1 relies on the observed returns. It is commonly used with PPO.
Generalised advantage estimation: an example
With lambda = 0.95 and gamma = 1, the advantage at each token blends the next few steps' value errors instead of just one step or the whole episode.
Related terms
More LLM training alignment terms
- Goodhart's law
- Group relative policy optimisation
- Helpful, honest and harmless
- Hugging Face TRL
- Identity preference optimisation
- InstructGPT
- Instruction backtranslation
- Instruction tuning
Where you learn it
Generalised advantage estimation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.