Stochastic Answer Networks for Machine Reading Comprehension

Abstract

We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determine the number of steps, the unique feature is the use of a kind of stochastic prediction dropout on the answer module (final layer) of the neural network during the training. We show that this simple trick improves robustness and achieves results competitive to the state-of-the-art on the Stanford Question Answering Dataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading COmprehension Dataset (MS MARCO).

arXiv comment11 pages, 5 figures, Accepted to ACL 2018

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Bidirectional Attention Flow for Machine Comprehension
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• Dynamic Coattention Networks For Question Answering
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Learning Recurrent Span Representations for Extractive Question Answering
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
• Stochastic Answer Networks for Natural Language Inference
• Making Neural QA as Simple as Possible but not Simpler
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
• Phase Conductor on Multi-layered Attentions for Machine Comprehension

Cited by in corpus (21)

• Recent Trends in Deep Learning Based Natural Language Processing
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• Deep Learning Based Text Classification: A Comprehensive Review
• The Natural Language Decathlon: Multitask Learning as Question Answering
• BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering
• Dynamically Fused Graph Network for Multi-hop Reasoning
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• U-Net: Machine Reading Comprehension with Unanswerable Questions
• KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension
• Data-Driven Methods for Solving Algebra Word Problems
• Adversarial TableQA: Attention Supervision for Question Answering on Tables
• Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading
• MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension
• Explicit Utilization of General Knowledge in Machine Reading Comprehension
• Do Multi-hop Readers Dream of Reasoning Chains?
• Cross-Lingual Transfer Learning for Question Answering
• Making Neural Machine Reading Comprehension Faster
• Integrated Triaging for Fast Reading Comprehension
• A Study of the Tasks and Models in Machine Reading Comprehension
• Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning
• Understanding in Artificial Intelligence