Simple Applications of BERT for Ad Hoc Document Retrieval

Abstract

Following recent successes in applying BERT to question answering, we explore simple applications to ad hoc document retrieval. This required confronting the challenge posed by documents that are typically longer than the length of input BERT was designed to handle. We address this issue by applying inference on sentences individually, and then aggregating sentence scores to produce document scores. Experiments on TREC microblog and newswire test collections show that our approach is simple yet effective, as we report the highest average precision on these datasets by neural approaches that we are aware of.

References in corpus (6)

• A Deep Relevance Matching Model for Ad-hoc Retrieval
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• End-to-End Open-Domain Question Answering with BERTserini
• Passage Re-ranking with BERT
• A Deep Look into Neural Ranking Models for Information Retrieval
• Evaluating Sentence-Level Relevance Feedback for High-Recall Information Retrieval

Cited by in corpus (48)

• BERTScore: Evaluating Text Generation with BERT
• Revisiting Few-sample BERT Fine-tuning
• Multi-Stage Document Ranking with BERT
• Information Retrieval: Recent Advances and Beyond
• Query Resolution for Conversational Search with Limited Supervision
• Sparse, Dense, and Attentional Representations for Text Retrieval
• How Context Affects Language Models' Factual Predictions
• Training Neural Response Selection for Task-Oriented Dialogue Systems
• Neural ranking models for document retrieval
• StruBERT: Structure-aware BERT for Table Search and Matching
• Diagnosing BERT with Retrieval Heuristics
• Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims
• Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning
• Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset
• B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval
• Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach
• A Deep Look into Neural Ranking Models for Information Retrieval
• Synthetic Document Generator for Annotation-free Layout Recognition
• Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature
• Query Focused Multi-Document Summarization with Distant Supervision
• Semantic Labeling Using a Deep Contextualized Language Model
• CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search
• Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
• Simplified TinyBERT: Knowledge Distillation for Document Retrieval
• News Article Retrieval in Context for Event-centric Narrative Creation
• Cross-Lingual Relevance Transfer for Document Retrieval
• Transformer Based Language Models for Similar Text Retrieval and Ranking
• Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering
• Multistage BiCross encoder for multilingual access to COVID-19 health information
• SQRQuerier: A Visual Querying Framework for Cross-national Survey Data Recycling
• Open-domain question classification and completion in conversational information search
• Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls
• On the Calibration and Uncertainty of Neural Learning to Rank Models
• Cross-Lingual Training with Dense Retrieval for Document Retrieval
• An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features
• Long Document Ranking with Query-Directed Sparse Transformer
• Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
• Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need
• GLOW : Global Weighted Self-Attention Network for Web Search
• An In-depth Analysis of Passage-Level Label Transfer for Contextual Document Ranking
• A neural document language modeling framework for spoken document retrieval
• Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
• LRG at TREC 2020: Document Ranking with XLNet-Based Models
• COVID-19: Comparative Analysis of Methods for Identifying Articles Related to Therapeutics and Vaccines without Using Labeled Data
• SAUCE: Truncated Sparse Document Signature Bit-Vectors for Fast Web-Scale Corpus Expansion
• Gender prediction using limited Twitter Data