Taking Class Imbalance Into Account in Open Set Recognition Evaluation

Abstract

In recent years Deep Neural Network-based systems are not only increasing in popularity but also receive growing user trust. However, due to the closed-world assumption of such systems, they cannot recognize samples from unknown classes and often induce an incorrect label with high confidence. Presented work looks at the evaluation of methods for Open Set Recognition, focusing on the impact of class imbalance, especially in the dichotomy between known and unknown samples. As an outcome of problem analysis, we present a set of guidelines for evaluation of methods in this field.

References in corpus (7)

• SMOTE: Synthetic Minority Over-sampling Technique
• Recent Advances in Open Set Recognition: A Survey
• Model Adaptation: Unsupervised Domain Adaptation without Source Data
• Towards Open World Recognition
• A Review of Generalized Zero-Shot Learning Methods
• Learning Open Set Network with Discriminative Reciprocal Points
• torchosr -- a PyTorch extension package for Open Set Recognition models evaluation in Python