Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering

Abstract

We address the problem of extractive question answering using document-level distant super-vision, pairing questions and relevant documents with answer strings. We compare previously used probability space and distant super-vision assumptions (assumptions on the correspondence between the weak answer string labels and possible answer mention spans). We show that these assumptions interact, and that different configurations provide complementary benefits. We demonstrate that a multi-objective model can efficiently combine the advantages of multiple assumptions and out-perform the best individual formulation. Our approach outperforms previous state-of-the-art models by 4.3 points in F1 on TriviaQA-Wiki and 1.7 points in Rouge-L on NarrativeQA summaries.

ACL2020

References in corpus (3)

• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Quasar: Datasets for Question Answering by Search and Reading
• R${}^3$: Reinforced Reader-Ranker for Open-Domain Question Answering

Cited by in corpus (6)

• Introspective Distillation for Robust Question Answering
• ReadTwice: Reading Very Large Documents with Memories
• UnitedQA: A Hybrid Approach for Open Domain Question Answering
• Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotation
• Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study
• DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization