AI engineering glossary · Reinforcement learning
What is bellman backup?
Bellman backup: A single update that replaces the estimate for one situation with the score received plus the discounted estimate of what follows. Repeated many times across all situations, such updates make estimates converge to the true values.
Bellman backup: an example
Updating V(s) from 0 to 1 + 0.9*V(s') = 1 + 0.9*5 = 5.5 is one backup.
Related terms
More Reinforcement learning terms
- Bellman equation
- Bellman optimality equation
- Bootstrapping
- Conservative Q-learning
- Contextual bandit
- Continuing task
- Count-based exploration
- Credit assignment
Where you learn it
Bellman backup is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.