Complementing Lexical Retrieval with Semantic Residual Embedding

Abstract

This paper presents CLEAR, a retrieval model that seeks to complement classical lexical exact-match models such as BM25 with semantic matching signals from a neural embedding matching model. CLEAR explicitly trains the neural embedding to encode language structures and semantics that lexical retrieval fails to capture with a novel residual-based embedding learning method. Empirical evaluations demonstrate the advantages of CLEAR over state-of-the-art retrieval models, and that it can substantially improve the end-to-end accuracy and efficiency of reranking pipelines.

arXiv commentECIR 2021

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• REALM: Retrieval-Augmented Language Model Pre-Training
• Deeper Text Understanding for IR with Contextual Neural Language Modeling
• Passage Re-ranking with BERT
• Document Expansion by Query Prediction
• Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval
• Pre-training Tasks for Embedding-based Large-scale Retrieval
• Overview of the TREC 2021 deep learning track
• Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach

Cited by in corpus (30)

• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Sparse, Dense, and Attentional Representations for Text Retrieval
• A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques
• Distilling Dense Representations for Ranking using Tightly-Coupled Teachers
• Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansion
• The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims
• CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typos
• Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluation
• Optimizing Guided Traversal for Fast Learned Sparse Retrieval
• Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and Efficiently
• SpaDE: Improving Sparse Representations using a Dual Document Encoder for First-stage Retrieval
• Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval
• Optimizing Dense Retrieval Model Training with Hard Negatives
• CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search
• Asyncval: A Toolkit for Asynchronously Validating Dense Retriever Checkpoints during Training
• Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls
• PGT: Pseudo Relevance Feedback Using a Graph-Based Transformer
• Hybrid and Collaborative Passage Reranking
• Interpreting Dense Retrieval as Mixture of Topics
• Contextualized Query Embeddings for Conversational Search
• DoSSIER@COLIEE 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval
• Dealing with Typos for BERT-based Passage Retrieval and Ranking
• Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance
• Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
• More Robust Dense Retrieval with Contrastive Dual Learning
• Extreme Multi-label Learning for Semantic Matching in Product Search