Simple Applications of BERT for Ad Hoc Document Retrieval

Abstract

Following recent successes in applying BERT to question answering, we explore simple applications to ad hoc document retrieval. This required confronting the challenge posed by documents that are typically longer than the length of input BERT was designed to handle. We address this issue by applying inference on sentences individually, and then aggregating sentence scores to produce document scores. Experiments on TREC microblog and newswire test collections show that our approach is simple yet effective, as we report the highest average precision on these datasets by neural approaches that we are aware of.

References in corpus (3)

• A Deep Relevance Matching Model for Ad-hoc Retrieval
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• A Deep Look into Neural Ranking Models for Information Retrieval

Cited by in corpus (29)

• Multi-Stage Document Ranking with BERT
• Query Resolution for Conversational Search with Limited Supervision
• How Context Affects Language Models' Factual Predictions
• Training Neural Response Selection for Task-Oriented Dialogue Systems
• StruBERT: Structure-aware BERT for Table Search and Matching
• Diagnosing BERT with Retrieval Heuristics
• Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims
• Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset
• Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach
• A Deep Look into Neural Ranking Models for Information Retrieval
• Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature
• Query Focused Multi-Document Summarization with Distant Supervision
• Semantic Labeling Using a Deep Contextualized Language Model
• CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search
• News Article Retrieval in Context for Event-centric Narrative Creation
• Transformer Based Language Models for Similar Text Retrieval and Ranking
• Cross-Lingual Relevance Transfer for Document Retrieval
• Open-domain question classification and completion in conversational information search
• An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
• On the Calibration and Uncertainty of Neural Learning to Rank Models
• Cross-Lingual Training with Dense Retrieval for Document Retrieval
• An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need
• LRG at TREC 2020: Document Ranking with XLNet-Based Models
• A neural document language modeling framework for spoken document retrieval
• COVID-19: Comparative Analysis of Methods for Identifying Articles Related to Therapeutics and Vaccines without Using Labeled Data
• Gender prediction using limited Twitter Data
• SAUCE: Truncated Sparse Document Signature Bit-Vectors for Fast Web-Scale Corpus Expansion