Expansion via Prediction of Importance with Contextualization

Abstract

The identification of relevance with little textual context is a primary challenge in passage retrieval. We address this problem with a representation-based ranking approach that: (1) explicitly models the importance of each term using a contextualized language model; (2) performs passage expansion by propagating the importance to similar terms; and (3) grounds the representations in the lexicon, making them interpretable. Passage representations can be pre-computed at index time to reduce query-time latency. We call our approach EPIC (Expansion via Prediction of Importance with Contextualization). We show that EPIC significantly outperforms prior importance-modeling and document expansion approaches. We also observe that the performance is additive with the current leading first-stage retrieval methods, further narrowing the gap between inexpensive and cost-prohibitive passage ranking approaches. Specifically, EPIC achieves a MRR@10 of 0.304 on the MS-MARCO passage ranking dataset with 78ms average query latency on commodity hardware. We also find that the latency is further reduced to 68ms by pruning document representations, with virtually no difference in effectiveness.

arXiv commentAccepted at SIGIR 2020 (short)

References in corpus (1)

• CEDR: Contextualized Embeddings for Document Ranking

Cited by in corpus (17)

• Information Retrieval: Recent Advances and Beyond
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations
• Evaluating Generative Ad Hoc Information Retrieval
• Query Embedding Pruning for Dense Retrieval
• ABNIRML: Analyzing the Behavior of Neural IR Models
• Adaptive Re-Ranking with a Corpus Graph
• Lexically-Accelerated Dense Retrieval
• Efficient Neural Ranking using Forward Indexes
• On Approximate Nearest Neighbour Selection for Multi-Stage Dense Retrieval
• Pairing Clustered Inverted Indexes with kNN Graphs for Fast Approximate Retrieval over Learned Sparse Representations
• Neural Passage Quality Estimation for Static Pruning
• Effective Inference-Free Retrieval for Learned Sparse Representations
• Exploring ${\ell }_0$ Sparsification for Inference-free Sparse Retrievers
• GazBy: Gaze-Based BERT Model to Incorporate Human Attention in Neural Information Retrieval
• Shallow Cross-Encoders for Low-Latency Retrieval
• Contextualization with SPLADE for High Recall Retrieval