Learning Discriminative Representations and Decision Boundaries for Open Intent Detection

Abstract

Open intent detection is a significant problem in natural language understanding, which aims to identify the unseen open intent while ensuring known intent identification performance. However, current methods face two major challenges. Firstly, they struggle to learn friendly representations to detect the open intent with prior knowledge of only known intents. Secondly, there is a lack of an effective approach to obtaining specific and compact decision boundaries for known intents. To address these issues, this paper presents an original framework called DA-ADB, which successively learns distance-aware intent representations and adaptive decision boundaries for open intent detection. Specifically, we first leverage distance information to enhance the distinguishing capability of the intent representations. Then, we design a novel loss function to obtain appropriate decision boundaries by balancing both empirical and open space risks. Extensive experiments demonstrate the effectiveness of the proposed distance-aware and boundary learning strategies. Compared to state-of-the-art methods, our framework achieves substantial improvements on three benchmark datasets. Furthermore, it yields robust performance with varying proportions of labeled data and known categories.

Accepted by IEEE/ACM TASLP

References in corpus (9)

• Energy-based Out-of-distribution Detection
• Adversarial Reciprocal Points Learning for Open Set Recognition
• Generalized Out-of-Distribution Detection: A Survey
• Deep Open Intent Classification with Adaptive Decision Boundary
• A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling
• Discovering New Intents with Deep Aligned Clustering
• MIntRec: A New Dataset for Multimodal Intent Recognition
• A Post-processing Method for Detecting Unknown Intent of Dialogue System via Pre-trained Deep Neural Network Classifier
• TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition

Cited by in corpus (1)

• A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery