Subword-augmented Embedding for Cloze Reading Comprehension

Abstract

Representation learning is the foundation of machine reading comprehension. In state-of-the-art models, deep learning methods broadly use word and character level representations. However, character is not naturally the minimal linguistic unit. In addition, with a simple concatenation of character and word embedding, previous models actually give suboptimal solution. In this paper, we propose to use subword rather than character for word embedding enhancement. We also empirically explore different augmentation strategies on subword-augmented embedding to enhance the cloze-style reading comprehension model reader. In detail, we present a reader that uses subword-level representation to augment word embedding with a short list to handle rare words effectively. A thorough examination is conducted to evaluate the comprehensive performance and generalization ability of the proposed reader. Experimental results show that the proposed approach helps the reader significantly outperform the state-of-the-art baselines on various public datasets.

arXiv commentProceedings of the 27th International Conference on Computational Linguistics (COLING 2018)

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• Teaching Machines to Read and Comprehend
• Character-Aware Neural Language Models
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• Machine Comprehension Using Match-LSTM and Answer Pointer
• Multi-Task Cross-Lingual Sequence Tagging from Scratch
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• Deep Enhanced Representation for Implicit Discourse Relation Recognition
• Gated Word-Character Recurrent Language Model
• Consensus Attention-based Neural Networks for Chinese Reading Comprehension
• Neural Word Segmentation Learning for Chinese
• Character-Word LSTM Language Models

Cited by in corpus (4)

• Lingke: A Fine-grained Multi-turn Chatbot for Customer Service
• Explicit Contextual Semantics for Text Comprehension
• SG-Net: Syntax-Guided Machine Reading Comprehension
• Chinese Pinyin Aided IME, Input What You Have Not Keystroked Yet