Ruminating Reader: Reasoning with Gated Multi-Hop Attention
arXiv:1704.07415
Abstract
To answer the question in machine comprehension (MC) task, the models need to establish the interaction between the question and the context. To tackle the problem that the single-pass model cannot reflect on and correct its answer, we present Ruminating Reader. Ruminating Reader adds a second pass of attention and a novel information fusion component to the Bi-Directional Attention Flow model (BiDAF). We propose novel layer structures that construct an query-aware context vector representation and fuse encoding representation with intermediate representation on top of BiDAF model. We show that a multi-hop attention mechanism can be applied to a bi-directional attention structure. In experiments on SQuAD, we find that the Reader outperforms the BiDAF baseline by a substantial margin, and matches or surpasses the performance of all other published systems.
10 pages, 6 figures
References in corpus (6)
- ADADELTA: An Adaptive Learning Rate Method
- Machine Comprehension Using Match-LSTM and Answer Pointer
- Learning Recurrent Span Representations for Extractive Question Answering
- Multi-Perspective Context Matching for Machine Comprehension
- End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension
- Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
Cited by in corpus (9)
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Adversarial Examples for Evaluating Reading Comprehension Systems
- Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
- Reinforced Mnemonic Reader for Machine Reading Comprehension
- Smarnet: Teaching Machines to Read and Comprehend Like Human
- Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
- Making Neural Machine Reading Comprehension Faster
- Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
- Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples