AI engineering glossary · Time series recsys
What is counterfactual evaluation?
Counterfactual evaluation: Estimating how a new recommendation or ranking method would have performed using only logs generated by an old method, correcting for the fact that the logs reflect the old method's choices. It reduces the need for costly online tests.
Counterfactual evaluation: an example
Estimate a new ranker's click rate from last month's logs using inverse propensity weights.
Related terms
More Time series recsys terms
Where you learn it
Counterfactual evaluation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.