Cascade Model-based Propensity Estimation for Counterfactual Learning to Rank

Abstract

Unbiased CLTR requires click propensities to compensate for the difference between user clicks and true relevance of search results via IPS. Current propensity estimation methods assume that user click behavior follows the PBM and estimate click propensities based on this assumption. However, in reality, user clicks often follow the CM, where users scan search results from top to bottom and where each next click depends on the previous one. In this cascade scenario, PBM-based estimates of propensities are not accurate, which, in turn, hurts CLTR performance. In this paper, we propose a propensity estimation method for the cascade scenario, called CM-IPS. We show that CM-IPS keeps CLTR performance close to the full-information performance in case the user clicks follow the CM, while PBM-based CLTR has a significant gap towards the full-information. The opposite is true if the user clicks follow PBM instead of the CM. Finally, we suggest a way to select between CM- and PBM-based propensity estimation methods based on historical user clicks.

4 pages, 2 figures, 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20)

Cited by in corpus (9)

• The Bandwagon Effect: Not Just Another Bias
• Mixture-Based Correction for Position and Trust Bias in Counterfactual Learning to Rank
• The Importance of Causality in Decision Making: A Perspective on Recommender Systems
• The Role of Relevance in Fair Ranking
• FairSort: Learning to Fair Rank for Personalized Recommendations in Two-Sided Platforms
• Click-Conversion Multi-Task Model with Position Bias Mitigation for Sponsored Search in eCommerce
• Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
• AIE: Auction Information Enhanced Framework for CTR Prediction in Online Advertising
• An Epistemic Position-Based Click Model: From Interactions to Epistemic Distributions of Relevance and Bias