AI engineering glossary · Reinforcement learning
What is overestimation bias?
Overestimation bias: The tendency of maximum-based value updates to produce estimates that are too high, because taking a maximum over noisy numbers picks up positive noise. It can mislead the learner toward actions that only look good by chance.
Overestimation bias: an example
If five actions all have true value 0 but noisy estimates, the max of the estimates is usually above 0.
Related terms
More Reinforcement learning terms
- Partially observable Markov decision process
- Policy evaluation
- Policy gradient theorem
- Policy improvement
- Policy iteration
- Potential-based reward shaping
- Prioritised experience replay
- Q-learning
Where you learn it
Overestimation bias is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.