Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling
arXiv:1601.01530
Abstract
Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling. In this paper, we first enhance LSTM-based sequence labeling to explicitly model label dependencies. Then we propose another enhancement to incorporate the global information spanning over the whole input sequence. The latter proposed method, encoder-labeler LSTM, first encodes the whole input sequence into a fixed length vector with the encoder LSTM, and then uses this encoded vector as the initial state of another LSTM for sequence labeling. Combining these methods, we can predict the label sequence with considering label dependencies and information of whole input sequence. In the experiments of a slot filling task, which is an essential component of natural language understanding, with using the standard ATIS corpus, we achieved the state-of-the-art F1-score of 95.66%.
Accepted in EMNLP 2016
References in corpus (5)
Cited by in corpus (9)
- Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
- Neural Models for Sequence Chunking
- Concurrent Activity Recognition with Multimodal CNN-LSTM Structure
- Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding
- Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents
- Encoder-decoder with Focus-mechanism for Sequence Labelling Based Spoken Language Understanding
- CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding
- Character-Level Feature Extraction with Densely Connected Networks
- Linguistically-Enriched and Context-Aware Zero-shot Slot Filling