Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation

Abstract

The findings on open-set recognition (OSR) show that models trained on classification datasets are capable of detecting unknown classes not encountered during the training process. Specifically, after training, the learned representations of known classes dissociate from the representations of the unknown class, facilitating OSR. In this paper, we investigate this emergent phenomenon by examining the relationship between the Jacobian norm of representations and the inter/intra-class learning dynamics. We provide a theoretical analysis, demonstrating that intra-class learning reduces the Jacobian norm for known class samples, while inter-class learning increases the Jacobian norm for unknown samples, even in the absence of direct exposure to any unknown sample. Overall, the discrepancy in the Jacobian norm between the known and unknown classes enables OSR. Based on this insight, which highlights the pivotal role of inter-class learning, we devise a marginal one-vs-rest (m-OvR) loss function that promotes strong inter-class separation. To further improve OSR performance, we integrate the m-OvR loss with additional strategies that maximize the Jacobian norm disparity. We present comprehensive experimental results that support our theoretical observations and demonstrate the efficacy of our proposed OSR approach.

arXiv commentAccepted to Pattern Recognition

References in corpus (16)

• mixup: Beyond Empirical Risk Minimization
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Recent Advances in Open Set Recognition: A Survey
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Deep Anomaly Detection Using Geometric Transformations
• Adversarial Reciprocal Points Learning for Open Set Recognition
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• Generalized Out-of-Distribution Detection: A Survey
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• Robust Large Margin Deep Neural Networks
• Collective Decision of One-vs-Rest Networks for Open Set Recognition
• The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods
• Deep Metric Learning with Spherical Embedding
• Maximum Class Separation as Inductive Bias in One Matrix