Effective Subword Segmentation for Text Comprehension

Abstract

Representation learning is the foundation of machine reading comprehension and inference. In state-of-the-art models, character-level representations have been broadly adopted to alleviate the problem of effectively representing rare or complex words. However, character itself is not a natural minimal linguistic unit for representation or word embedding composing due to ignoring the linguistic coherence of consecutive characters inside word. This paper presents a general subword-augmented embedding framework for learning and composing computationally-derived subword-level representations. We survey a series of unsupervised segmentation methods for subword acquisition and different subword-augmented strategies for text understanding, showing that subword-augmented embedding significantly improves our baselines in various types of text understanding tasks on both English and Chinese benchmarks.

arXiv commentAccepted by IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• Bidirectional Attention Flow for Machine Comprehension
• Training Very Deep Networks
• Enhanced LSTM for Natural Language Inference
• Character-Aware Neural Language Models
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• Multi-Task Cross-Lingual Sequence Tagging from Scratch
• Compositional Morphology for Word Representations and Language Modelling
• Modeling Multi-turn Conversation with Deep Utterance Aggregation
• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• Dual Co-Matching Network for Multi-choice Reading Comprehension
• One-shot Learning for Question-Answering in Gaokao History Challenge
• Explicit Contextual Semantics for Text Comprehension
• A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique
• Dependency or Span, End-to-End Uniform Semantic Role Labeling
• SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings