AI engineering glossary · Reinforcement learning
What is importance sampling ratio?
Importance sampling ratio: The ratio between the probability that the target decision rule and the data-collecting rule would each choose an action. Multiplying returns by it corrects for the mismatch when learning from experience gathered by another rule. Formula: rho = pi(a given s) / b(a given s).
Importance sampling ratio: an example
If the new rule picks an action with probability 0.6 and the old one with 0.2, that sample is weighted 3.
Related terms
- Off-policy
- Off-policy evaluation
- Proximal policy optimisation
- Monte Carlo methods (reinforcement learning)
More Reinforcement learning terms
- Intrinsic motivation
- Inverse reinforcement learning
- Markov decision process
- Maximum entropy reinforcement learning
- Model predictive control
- Model-based
- Model-free
- Monte Carlo methods (reinforcement learning)
Where you learn it
Importance sampling ratio is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.