AI engineering glossary · Reinforcement learning
What is temporal-difference learning?
Temporal-difference learning (also called TD learning): A family of methods that update a value estimate using the difference between a prediction and a slightly better prediction one step later, rather than waiting for the run to end. It learns online, step by step, and mixes ideas from sampling and from recursion.
Temporal-difference learning: an example
If V(s) = 5, reward 1, next value 6 and gamma = 0.9, the target is 1 + 0.9*6 = 6.4, so V(s) is nudged up towards 6.4.
Related terms
More Reinforcement learning terms
- Terminal state
- Thompson sampling
- Transition probability
- Twin delayed DDPG
- Upper confidence bound
- Value iteration
- World model
- A3C
Where you learn it
Temporal-difference learning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.