M2IOSR: Maximal Mutual Information Open Set Recognition

Abstract

In this work, we aim to address the challenging task of open set recognition (OSR). Many recent OSR methods rely on auto-encoders to extract class-specific features by a reconstruction strategy, requiring the network to restore the input image on pixel-level. This strategy is commonly over-demanding for OSR since class-specific features are generally contained in target objects, not in all pixels. To address this shortcoming, here we discard the pixel-level reconstruction strategy and pay more attention to improving the effectiveness of class-specific feature extraction. We propose a mutual information-based method with a streamlined architecture, Maximal Mutual Information Open Set Recognition (M2IOSR). The proposed M2IOSR only uses an encoder to extract class-specific features by maximizing the mutual information between the given input and its latent features across multiple scales. Meanwhile, to further reduce the open space risk, latent features are constrained to class conditional Gaussian distributions by a KL-divergence loss function. In this way, a strong function is learned to prevent the network from mapping different observations to similar latent features and help the network extract class-specific features with desired statistical characteristics. The proposed method significantly improves the performance of baselines and achieves new state-of-the-art results on several benchmarks consistently.

References in corpus (8)

• Learning deep representations by mutual information estimation and maximization
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• MINE: Mutual Information Neural Estimation
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• Learning Open Set Network with Discriminative Reciprocal Points
• On Variational Bounds of Mutual Information
• Conditional Gaussian Distribution Learning for Open Set Recognition

Cited by in corpus (3)

• Generalized Out-of-Distribution Detection: A Survey
• Knowledge-aware Deep Framework for Collaborative Skin Lesion Segmentation and Melanoma Recognition
• Calibrating Class Activation Maps for Long-Tailed Visual Recognition