Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

Abstract

BERT model has been successfully applied to open-domain QA tasks. However, previous work trains BERT by viewing passages corresponding to the same question as independent training instances, which may cause incomparable scores for answers from different passages. To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages. In addition, we find that splitting articles into passages with the length of 100 words by sliding window improves performance by 4%. By leveraging a passage ranker to select high-quality passages, multi-passage BERT gains additional 2%. Experiments on four standard benchmarks showed that our multi-passage BERT outperforms all state-of-the-art models on all benchmarks. In particular, on the OpenSQuAD dataset, our model gains 21.4% EM and 21.5% $F_1$ over all non-BERT models, and 5.8% EM and 6.5% $F_1$ over BERT-based models.

To appear in EMNLP 2019

References in corpus (8)

• Bidirectional Attention Flow for Machine Comprehension
• End-to-End Open-Domain Question Answering with BERTserini
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Quasar: Datasets for Question Answering by Search and Reading
• Multi-Perspective Context Matching for Machine Comprehension
• A BERT Baseline for the Natural Questions
• A Unified Query-based Generative Model for Question Generation and Question Answering
• Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation

Cited by in corpus (7)

• Big Bird: Transformers for Longer Sequences
• Exploring the State of the Art in Legal QA Systems
• Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
• Knowledge-Aided Open-Domain Question Answering
• SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
• AutoKG: Constructing Virtual Knowledge Graphs from Unstructured Documents for Question Answering
• CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering