A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques

Abstract

Recent developments in representational learning for information retrieval can be organized in a conceptual framework that establishes two pairs of contrasts: sparse vs. dense representations and unsupervised vs. learned representations. Sparse learned representations can further be decomposed into expansion and term weighting components. This framework allows us to understand the relationship between recently proposed techniques such as DPR, ANCE, DeepCT, DeepImpact, and COIL, and furthermore, gaps revealed by our analysis point to "low hanging fruit" in terms of techniques that have yet to be explored. We present a novel technique dubbed "uniCOIL", a simple extension of COIL that achieves to our knowledge the current state-of-the-art in sparse retrieval on the popular MS MARCO passage ranking dataset. Our implementation using the Anserini IR toolkit is built on the Lucene search library and thus fully compatible with standard inverted indexes.

References in corpus (8)

• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval
• Pretrained Transformers for Text Ranking: BERT and Beyond
• RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
• Complementing Lexical Retrieval with Semantic Residual Embedding
• A Replication Study of Dense Passage Retriever
• Efficient Passage Retrieval with Hashing for Open-domain Question Answering
• Learning Passage Impacts for Inverted Indexes

Cited by in corpus (14)

• Information Retrieval: Recent Advances and Beyond
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansion
• Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluation
• To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
• Optimizing Guided Traversal for Fast Learned Sparse Retrieval
• SpaDE: Improving Sparse Representations using a Dual Document Encoder for First-stage Retrieval
• Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval
• SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes
• On the Interpolation of Contextualized Term-based Ranking with BM25 for Query-by-Example Retrieval
• Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach
• An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
• Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
• Searching Personal Collections