Differentiable Unbiased Online Learning to Rank

Abstract

Online Learning to Rank (OLTR) methods optimize rankers based on user interactions. State-of-the-art OLTR methods are built specifically for linear models. Their approaches do not extend well to non-linear models such as neural networks. We introduce an entirely novel approach to OLTR that constructs a weighted differentiable pairwise loss after each interaction: Pairwise Differentiable Gradient Descent (PDGD). PDGD breaks away from the traditional approach that relies on interleaving or multileaving and extensive sampling of models to estimate gradients. Instead, its gradient is based on inferring preferences between document pairs from user clicks and can optimize any differentiable model. We prove that the gradient of PDGD is unbiased w.r.t. user document pair preferences. Our experiments on the largest publicly available Learning to Rank (LTR) datasets show considerable and significant improvements under all levels of interaction noise. PDGD outperforms existing OLTR methods both in terms of learning speed as well as final convergence. Furthermore, unlike previous OLTR methods, PDGD also allows for non-linear models to be optimized effectively. Our results show that using a neural network leads to even better performance at convergence than a linear model. In summary, PDGD is an efficient and unbiased OLTR approach that provides a better user experience than previously possible.

Conference on Information and Knowledge Management 2018

References in corpus (3)

• On Application of Learning to Rank for E-Commerce Search
• Balancing Speed and Quality in Online Learning to Rank for Information Retrieval
• Sensitive and Scalable Online Evaluation with Theoretical Guarantees

Cited by in corpus (20)

• Correcting for Selection Bias in Learning-to-rank Systems
• Unifying Online and Counterfactual Learning to Rank
• To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions
• Policy-Aware Unbiased Learning to Rank for Top-k Rankings
• Cascade Model-based Propensity Estimation for Counterfactual Learning to Rank
• Variance Reduction in Gradient Exploration for Online Learning to Rank
• Cascading Hybrid Bandits: Online Learning to Rank for Relevance and Diversity
• Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for Unbiased Learning to Rank
• PairRank: Online Pairwise Learning to Rank by Divide-and-Conquer
• Mixture-Based Correction for Position and Trust Bias in Counterfactual Learning to Rank
• The Archive Query Log: Mining Millions of Search Result Pages of Hundreds of Search Engines from 25 Years of Web Archives
• Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach
• The Role of Relevance in Fair Ranking
• Robust Generalization and Safe Query-Specialization in Counterfactual Learning to Rank
• On the Impact of Outlier Bias on User Clicks
• Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
• Meta Learning to Rank for Sparsely Supervised Queries
• Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study
• A General Framework for Pairwise Unbiased Learning to Rank
• Exposure-Based Reinforcement Learning to Rank