Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering

Abstract

Recently, open-domain question answering (QA) has been combined with machine comprehension models to find answers in a large knowledge source. As open-domain QA requires retrieving relevant documents from text corpora to answer questions, its performance largely depends on the performance of document retrievers. However, since traditional information retrieval systems are not effective in obtaining documents with a high probability of containing answers, they lower the performance of QA systems. Simply extracting more documents increases the number of irrelevant documents, which also degrades the performance of QA systems. In this paper, we introduce Paragraph Ranker which ranks paragraphs of retrieved documents for a higher answer recall with less noise. We show that ranking paragraphs and aggregating answers using Paragraph Ranker improves performance of open-domain QA pipeline on the four open-domain QA datasets by 7.8% on average.

arXiv commentEMNLP 2018

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Bidirectional Attention Flow for Machine Comprehension
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Dynamic Coattention Networks For Question Answering
• Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering
• Reading Wikipedia to Answer Open-Domain Questions
• R${}^3$: Reinforced Reader-Ranker for Open-Domain Question Answering
• Learning Natural Language Inference with LSTM
• An Exploration of Approaches to Integrating Neural Reranking Models in Multi-Stage Ranking Architectures

Cited by in corpus (7)

• Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
• Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering
• EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System
• SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
• Talk to Papers: Bringing Neural Question Answering to Academic Search
• Recent Advances in Automated Question Answering In Biomedical Domain
• MIX : a Multi-task Learning Approach to Solve Open-Domain Question Answering