MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension

Abstract

Machine comprehension(MC) style question answering is a representative problem in natural language processing. Previous methods rarely spend time on the improvement of encoding layer, especially the embedding of syntactic information and name entity of the words, which are very crucial to the quality of encoding. Moreover, existing attention methods represent each query word as a vector or use a single vector to represent the whole query sentence, neither of them can handle the proper weight of the key words in query sentence. In this paper, we introduce a novel neural network architecture called Multi-layer Embedding with Memory Network(MEMEN) for machine reading task. In the encoding layer, we employ classic skip-gram model to the syntactic and semantic information of the words to train a new kind of embedding layer. We also propose a memory network of full-orientation matching of the query and passage to catch more pivotal information. Experiments show that our model has competitive results both from the perspectives of precision and efficiency in Stanford Question Answering Dataset(SQuAD) among all published results and achieves the state-of-the-art results on TriviaQA dataset.

References in corpus (11)

• ADADELTA: An Adaptive Learning Rate Method
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• Dynamic Coattention Networks For Question Answering
• Machine Comprehension Using Match-LSTM and Answer Pointer
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• Multi-Perspective Context Matching for Machine Comprehension
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Who did What: A Large-Scale Person-Centered Cloze Dataset
• Structural Embedding of Syntactic Trees for Machine Comprehension

Cited by in corpus (28)

• The Natural Language Decathlon: Multitask Learning as Question Answering
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Incorporating External Knowledge to Answer Open-Domain Visual Questions with Dynamic Memory Networks
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• A Taxonomy for Neural Memory Networks
• Neural Machine Reading Comprehension: Methods and Trends
• Efficient and Robust Question Answering from Minimal Context over Documents
• A Survey on Neural Machine Reading Comprehension
• The Effect of Natural Distribution Shift on Question Answering Models
• QA4IE: A Question Answering based Framework for Information Extraction
• Phase Conductor on Multi-layered Attentions for Machine Comprehension
• Smarnet: Teaching Machines to Read and Comprehend Like Human
• Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
• Explicit Utilization of General Knowledge in Machine Reading Comprehension
• HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering
• Making Neural Machine Reading Comprehension Faster
• Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension
• Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data
• Reinforced Dynamic Reasoning for Conversational Question Generation
• Multi-Mention Learning for Reading Comprehension with Neural Cascades
• Keyword-based Query Comprehending via Multiple Optimized-Demand Augmentation
• Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference
• Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System
• A Study of the Tasks and Models in Machine Reading Comprehension
• Question-Aware Sentence Gating Networks for Question and Answering
• Improved Dynamic Memory Network for Dialogue Act Classification with Adversarial Training
• Bi-Decoder Augmented Network for Neural Machine Translation