AI engineering glossary · Reinforcement learning
What is score function estimator?
Score function estimator (also called Likelihood-ratio trick, Log-derivative trick): A technique for estimating the gradient of an expectation over random choices by using grad p = p * grad log p. It lets gradients flow through sampling steps that cannot be differentiated directly.
Score function estimator: an example
The gradient of E[f(x)] over samples x from p is estimated by the mean of f(x) * grad log p(x).
Related terms
More Reinforcement learning terms
- Self-play
- Sim-to-real transfer
- Softmax exploration
- Sparse reward
- State
- Stochastic policy
- Target network
- TD error
Where you learn it
Score function estimator is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.