AI engineering glossary · Reinforcement learning
What is bellman optimality equation?
Bellman optimality equation: The recursive condition satisfied by the best possible values: the optimal value of a situation is the maximum over choices of immediate score plus discounted optimal value of the next situation. Formula: Q*(s,a) = E[r + gamma * max over a' of Q*(s',a')]. Q-learning aims to solve it.
Bellman optimality equation: an example
If from s' the best two options are worth 3 and 8, then the target for the previous step with r = 1 and gamma = 0.9 is 1 + 0.9*8 = 8.2.
Related terms
More Reinforcement learning terms
- Bootstrapping
- Conservative Q-learning
- Contextual bandit
- Continuing task
- Count-based exploration
- Credit assignment
- Curiosity-driven exploration
- DAgger
Where you learn it
Bellman optimality equation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.