Taking the Counterfactual Online: Efficient and Unbiased Online Evaluation for Ranking

Abstract

Counterfactual evaluation can estimate Click-Through-Rate (CTR) differences between ranking systems based on historical interaction data, while mitigating the effect of position bias and item-selection bias. We introduce the novel Logging-Policy Optimization Algorithm (LogOpt), which optimizes the policy for logging data so that the counterfactual estimate has minimal variance. As minimizing variance leads to faster convergence, LogOpt increases the data-efficiency of counterfactual estimation. LogOpt turns the counterfactual approach - which is indifferent to the logging policy - into an online approach, where the algorithm decides what rankings to display. We prove that, as an online evaluation method, LogOpt is unbiased w.r.t. position and item-selection bias, unlike existing interleaving methods. Furthermore, we perform large-scale experiments by simulating comparisons between thousands of rankers. Our results show that while interleaving methods make systematic errors, LogOpt is as efficient as interleaving without being biased.

arXiv commentICTIR 2020

References in corpus (6)

• Introducing LETOR 4.0 Datasets
• Unbiased Learning to Rank with Unbiased Propensity Estimation
• Estimating Position Bias without Intrusive Interventions
• Correcting for Selection Bias in Learning-to-rank Systems
• Policy-Aware Unbiased Learning to Rank for Top-k Rankings
• Intervention Harvesting for Context-Dependent Examination-Bias Estimation

Cited by in corpus (8)

• Unifying Online and Counterfactual Learning to Rank
• Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank
• Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization
• An Offline Metric for the Debiasedness of Click Models
• Learning from User Interactions with Rankings: A Unification of the Field
• Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
• Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness
• Exposure-Based Reinforcement Learning to Rank