CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots
arXiv:1909.08705
Abstract
Natural Language Understanding (NLU) is a core component of dialog systems. It typically involves two tasks - intent classification (IC) and slot labeling (SL), which are then followed by a dialogue management (DM) component. Such NLU systems cater to utterances in isolation, thus pushing the problem of context management to DM. However, contextual information is critical to the correct prediction of intents and slots in a conversation. Prior work on contextual NLU has been limited in terms of the types of contextual signals used and the understanding of their impact on the model. In this work, we propose a context-aware self-attentive NLU (CASA-NLU) model that uses multiple signals, such as previous intents, slots, dialog acts and utterances over a variable context window, in addition to the current user utterance. CASA-NLU outperforms a recurrent contextual NLU baseline on two conversational datasets, yielding a gain of up to 7% on the IC task for one of the datasets. Moreover, a non-contextual variant of CASA-NLU achieves state-of-the-art performance for IC task on standard public datasets - Snips and ATIS.
arXiv commentTo appear at EMNLP 2019
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Convolutional Sequence to Sequence Learning
- Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
- DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
- Global-Locally Self-Attentive Dialogue State Tracker
- Joint Slot Filling and Intent Detection via Capsule Neural Networks
- Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networks