Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling

Abstract

Attention-based encoder-decoder neural network models have recently shown promising results in machine translation and speech recognition. In this work, we propose an attention-based neural network model for joint intent detection and slot filling, both of which are critical steps for many speech understanding and dialog systems. Unlike in machine translation and speech recognition, alignment is explicit in slot filling. We explore different strategies in incorporating this alignment information to the encoder-decoder framework. Learning from the attention mechanism in encoder-decoder model, we further propose introducing attention to the alignment-based RNN models. Such attentions provide additional information to the intent classification and slot label prediction. Our independent task models achieve state-of-the-art intent detection error rate and slot filling F1 score on the benchmark ATIS task. Our joint training model further obtains 0.56% absolute (23.8% relative) error reduction on intent detection and 0.23% absolute gain on slot filling over the independent task models.

Accepted at Interspeech 2016

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Recurrent Neural Network Regularization
• Recurrent Neural Networks with External Memory for Language Understanding
• Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling

Cited by in corpus (10)

• Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
• Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
• Intent Detection with WikiHow
• Alquist 3.0: Alexa Prize Bot Using Conversational Knowledge Graph
• A Syllable-Structured, Contextually-Based Conditionally Generation of Chinese Lyrics
• Seq2Seq and Multi-Task Learning for joint intent and content extraction for domain specific interpreters
• Confusion2vec 2.0: Enriching Ambiguous Spoken Language Representations with Subwords
• Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding
• Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling
• Know thy corpus! Robust methods for digital curation of Web corpora