Ruminating Reader: Reasoning with Gated Multi-Hop Attention
arXiv:1704.07415
Abstract
To answer the question in machine comprehension (MC) task, the models need to establish the interaction between the question and the context. To tackle the problem that the single-pass model cannot reflect on and correct its answer, we present Ruminating Reader. Ruminating Reader adds a second pass of attention and a novel information fusion component to the Bi-Directional Attention Flow model (BiDAF). We propose novel layer structures that construct an query-aware context vector representation and fuse encoding representation with intermediate representation on top of BiDAF model. We show that a multi-hop attention mechanism can be applied to a bi-directional attention structure. In experiments on SQuAD, we find that the Reader outperforms the BiDAF baseline by a substantial margin, and matches or surpasses the performance of all other published systems.
arXiv comment10 pages, 6 figures
References in corpus (16)
- ADADELTA: An Adaptive Learning Rate Method
- Teaching Machines to Read and Comprehend
- Bidirectional Attention Flow for Machine Comprehension
- Character-Aware Neural Language Models
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Highway Networks
- The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
- Dynamic Coattention Networks For Question Answering
- Machine Comprehension Using Match-LSTM and Answer Pointer
- Learning Recurrent Span Representations for Extractive Question Answering
- A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
- Multi-Perspective Context Matching for Machine Comprehension
- Words or Characters? Fine-grained Gating for Reading Comprehension
- Making Neural QA as Simple as Possible but not Simpler
- End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension
- Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
Cited by in corpus (9)
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Adversarial Examples for Evaluating Reading Comprehension Systems
- Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
- Reinforced Mnemonic Reader for Machine Reading Comprehension
- Smarnet: Teaching Machines to Read and Comprehend Like Human
- Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
- Making Neural Machine Reading Comprehension Faster
- Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
- Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples