Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking

Abstract

An emerging recipe for achieving state-of-the-art effectiveness in neural document re-ranking involves utilizing large pre-trained language models - e.g., BERT - to evaluate all individual passages in the document and then aggregating the outputs by pooling or additional Transformer layers. A major drawback of this approach is high query latency due to the cost of evaluating every passage in the document with BERT. To make matters worse, this high inference cost and latency varies based on the length of the document, with longer documents requiring more time and computation. To address this challenge, we adopt an intra-document cascading strategy, which prunes passages of a candidate document using a less expensive model, called ESM, before running a scoring model that is more expensive and effective, called ETM. We found it best to train ESM (short for Efficient Student Model) via knowledge distillation from the ETM (short for Effective Teacher Model) e.g., BERT. This pruning allows us to only run the ETM model on a smaller set of passages whose size does not vary by document length. Our experiments on the MS MARCO and TREC Deep Learning Track benchmarks suggest that the proposed Intra-Document Cascaded Ranking Model (IDCM) leads to over 400% lower query latency by providing essentially the same effectiveness as the state-of-the-art BERT-based document ranking models.

arXiv commentAccepted at SIGIR 2021 (Full Paper Track)

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Longformer: The Long-Document Transformer
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• Deeper Text Understanding for IR with Contextual Neural Language Modeling
• Passage Re-ranking with BERT
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• Document Expansion by Query Prediction
• Multi-Stage Document Ranking with BERT
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Overview of the TREC 2021 deep learning track
• Conformer-Kernel with Query Term Independence for Document Retrieval

Cited by in corpus (5)

• Retrieval-Enhanced Machine Learning
• Adaptive Re-Ranking with a Corpus Graph
• The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
• Efficient Neural Ranking using Forward Indexes
• Towards Proactively Forecasting Sentence-Specific Information Popularity within Online News Documents