Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning
arXiv:2105.14289
Abstract
Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. A key challenge of OOD detection is to learn discriminative semantic features. Traditional cross-entropy loss only focuses on whether a sample is correctly classified, and does not explicitly distinguish the margins between categories. In this paper, we propose a supervised contrastive learning objective to minimize intra-class variance by pulling together in-domain intents belonging to the same class and maximize inter-class variance by pushing apart samples from different classes. Besides, we employ an adversarial augmentation mechanism to obtain pseudo diverse views of a sample in the latent space. Experiments on two public datasets prove the effectiveness of our method capturing discriminative representations for OOD detection.
arXiv commentAccepted by ACL2021
References in corpus (6)
- Adam: A Method for Stochastic Optimization
- A Simple Framework for Contrastive Learning of Visual Representations
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Adversarial Examples for Evaluating Reading Comprehension Systems
- Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systems