Recurrent Neural Networks with External Memory for Language Understanding
arXiv:1506.00195
Abstract
Recurrent Neural Networks (RNNs) have become increasingly popular for the task of language understanding. In this task, a semantic tagger is deployed to associate a semantic label to each word in an input sequence. The success of RNN may be attributed to its ability to memorize long-term dependence that relates the current-time semantic label prediction to the observations many time instances away. However, the memory capacity of simple RNNs is limited because of the gradient vanishing and exploding problem. We propose to use an external memory to improve memorization capability of RNNs. We conducted experiments on the ATIS dataset, and observed that the proposed model was able to achieve the state-of-the-art results. We compare our proposed model with alternative models and report analysis results that may provide insights for future research.
submitted to Interspeech 2015
References in corpus (3)
Cited by in corpus (12)
- Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
- Visual Question Answering: A Survey of Methods and Datasets
- Neural Models for Sequence Chunking
- Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling
- Visual Question Answering with Memory-Augmented Networks
- Neural Random-Access Machines
- A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling
- Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN
- Character-Level Feature Extraction with Densely Connected Networks
- Deep Cascade Multi-task Learning for Slot Filling in Online Shopping Assistant
- A New Concept of Deep Reinforcement Learning based Augmented General Sequence Tagging System
- User Information Augmented Semantic Frame Parsing using Coarse-to-Fine Neural Networks