AI engineering glossary · Reinforcement learning
What is TD error?
TD error (also called Temporal-difference error): The gap between a value estimate and the improved one-step estimate built from the received score and the next situation's value. Formula: delta = r + gamma * V(s') - V(s). Its size and sign drive the update and, in actor-critic methods, act as a learning signal for the actor.
TD error: an example
With r = 1, gamma = 0.9, V(s') = 6 and V(s) = 5, delta = 1 + 5.4 - 5 = 1.4.
Related terms
More Reinforcement learning terms
- TD(0)
- TD(lambda)
- Temporal-difference learning
- Terminal state
- Thompson sampling
- Transition probability
- Twin delayed DDPG
- Upper confidence bound
Where you learn it
TD error is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.