Dynamic Fusion Networks for Machine Reading Comprehension

Abstract

This paper presents a novel neural model - Dynamic Fusion Network (DFN), for machine reading comprehension (MRC). DFNs differ from most state-of-the-art models in their use of a dynamic multi-strategy attention process, in which passages, questions and answer candidates are jointly fused into attention vectors, along with a dynamic multi-step reasoning module for generating answers. With the use of reinforcement learning, for each input sample that consists of a question, a passage and a list of candidate answers, an instance of DFN with a sample-specific network architecture can be dynamically constructed by determining what attention strategy to apply and how many reasoning steps to take. Experiments show that DFNs achieve the best result reported on RACE, a challenging MRC dataset that contains real human reading questions in a wide variety of types. A detailed empirical analysis also demonstrates that DFNs can produce attention vectors that summarize information from questions, passages and answer candidates more effectively than other popular MRC models.

arXiv comment13 pages, 5 figures, 5 tables

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Playing Atari with Deep Reinforcement Learning
• Teaching Machines to Read and Comprehend
• Bidirectional Attention Flow for Machine Comprehension
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Dynamic Coattention Networks For Question Answering
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• RACE: Large-scale ReAding Comprehension Dataset From Examinations
• Reading Wikipedia to Answer Open-Domain Questions
• NewsQA: A Machine Comprehension Dataset
• Making Neural QA as Simple as Possible but not Simpler
• Learning to Design Games: Strategic Environments in Reinforcement Learning

Cited by in corpus (10)

• Multi-range Reasoning for Machine Comprehension
• Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions
• A Co-Matching Model for Multi-choice Reading Comprehension
• Improving Question Answering by Commonsense-Based Pre-Training
• Generating Distractors for Reading Comprehension Questions from Real Examinations
• Incorporating Structured Commonsense Knowledge in Story Completion
• Evidence Sentence Extraction for Machine Reading Comprehension
• Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
• Large-scale Cloze Test Dataset Created by Teachers
• Knowledge Efficient Deep Learning for Natural Language Processing