Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach

Abstract

In this paper we study how to effectively exploit implicit feedback in Dense Retrievers (DRs). We consider the specific case in which click data from a historic click log is available as implicit feedback. We then exploit such historic implicit interactions to improve the effectiveness of a DR. A key challenge that we study is the effect that biases in the click signal, such as position bias, have on the DRs. To overcome the problems associated with the presence of such bias, we propose the Counterfactual Rocchio (CoRocchio) algorithm for exploiting implicit feedback in Dense Retrievers. We demonstrate both theoretically and empirically that dense query representations learnt with CoRocchio are unbiased with respect to position bias and lead to higher retrieval effectiveness. We make available the implementations of the proposed methods and the experimental framework, along with all results at https://github.com/ielab/Counterfactual-DR.

Full paper, accepted at SGIR2022

References in corpus (7)

• Overview of the TREC 2020 deep learning track
• Estimating Position Bias without Intrusive Interventions
• A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques
• To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions
• Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansion
• Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations
• Sensitive and Scalable Online Evaluation with Theoretical Guarantees