Multi-range Reasoning for Machine Comprehension

Abstract

We propose MRU (Multi-Range Reasoning Units), a new fast compositional encoder for machine comprehension (MC). Our proposed MRU encoders are characterized by multi-ranged gating, executing a series of parameterized contract-and-expand layers for learning gating vectors that benefit from long and short-term dependencies. The aims of our approach are as follows: (1) learning representations that are concurrently aware of long and short-term context, (2) modeling relationships between intra-document blocks and (3) fast and efficient sequence encoding. We show that our proposed encoder demonstrates promising results both as a standalone encoder and as well as a complementary building block. We conduct extensive experiments on three challenging MC datasets, namely RACE, SearchQA and NarrativeQA, achieving highly competitive performance on all. On the RACE benchmark, our model outperforms DFN (Dynamic Fusion Networks) by 1.5%-6% without using any recurrent or convolution layers. Similarly, we achieve competitive performance relative to AMANDA on the SearchQA benchmark and BiDAF on the NarrativeQA benchmark without using any LSTM/GRU layers. Finally, incorporating MRU encoders with standard BiLSTM architectures further improves performance, achieving state-of-the-art results.

References in corpus (5)

• Machine Comprehension Using Match-LSTM and Answer Pointer
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Neural Machine Translation in Linear Time
• R${}^3$: Reinforced Reader-Ranker for Open-Domain Question Answering
• Dynamic Fusion Networks for Machine Reading Comprehension

Cited by in corpus (9)

• Option Comparison Network for Multiple-choice Reading Comprehension
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Dual Co-Matching Network for Multi-choice Reading Comprehension
• Commonsense for Generative Multi-Hop Question Answering Tasks
• Attention-Guided Answer Distillation for Machine Reading Comprehension
• Generating Distractors for Reading Comprehension Questions from Real Examinations
• Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering
• Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian
• DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension