Energy-based Unknown Intent Detection with Data Manipulation

Abstract

Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. In this paper, we propose using energy scores for this task as the energy score is theoretically aligned with the density of the input and can be derived from any classifier. However, high-quality OOD utterances are required during the training stage in order to shape the energy gap between OOD and in-distribution (IND), and these utterances are difficult to collect in practice. To tackle this problem, we propose a data manipulation framework to Generate high-quality OOD utterances with importance weighTs (GOT). Experimental results show that the energy-based detector fine-tuned by GOT can achieve state-of-the-art results on two benchmark datasets.

arXiv comment10 pages, 4 figures, accepted by Findings of ACL-IJCNLP 2021

References in corpus (8)

• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Understanding Black-box Predictions via Influence Functions
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Generative and Discriminative Text Classification with Recurrent Neural Networks
• Out-of-domain Detection for Natural Language Understanding in Dialog Systems
• Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog