Words or Characters? Fine-grained Gating for Reading Comprehension
arXiv:1611.01724
Abstract
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension. We present a fine-grained gating mechanism to dynamically combine word-level and character-level representations based on properties of the words. We also extend the idea of fine-grained gating to modeling the interaction between questions and paragraphs for reading comprehension. Experiments show that our approach can improve the performance on reading comprehension tasks, achieving new state-of-the-art results on the Children's Book Test dataset. To demonstrate the generality of our gating mechanism, we also show improved results on a social media tag prediction task.
arXiv commentAccepted as a conference paper at ICLR 2017
References in corpus (10)
- Character-Aware Neural Language Models
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
- Machine Comprehension Using Match-LSTM and Answer Pointer
- Multi-Task Cross-Lingual Sequence Tagging from Scratch
- Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation
- A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
- Review Networks for Caption Generation
- Gated Word-Character Recurrent Language Model
- End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension
Cited by in corpus (15)
- ReasoNet: Learning to Stop Reading in Machine Comprehension
- Differentiable Learning of Logical Rules for Knowledge Base Reasoning
- CharBERT: Character-aware Pre-trained Language Model
- Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
- Making Neural QA as Simple as Possible but not Simpler
- Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
- Neural Machine Reading Comprehension: Methods and Trends
- Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus
- Ruminating Reader: Reasoning with Gated Multi-Hop Attention
- Making Neural Machine Reading Comprehension Faster
- Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
- Contextual Recurrent Units for Cloze-style Reading Comprehension
- Pay More Attention - Neural Architectures for Question-Answering
- Question-Aware Sentence Gating Networks for Question and Answering
- CAESAR: Context Awareness Enabled Summary-Attentive Reader