OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data
arXiv:2107.08943
Abstract
Semi-supervised learning (SSL) has been a powerful strategy to incorporate few labels in learning better representations. In this paper, we focus on a practical scenario that one aims to apply SSL when unlabeled data may contain out-of-class samples - those that cannot have one-hot encoded labels from a closed-set of classes in label data, i.e., the unlabeled data is an open-set. Specifically, we introduce OpenCoS, a simple framework for handling this realistic semi-supervised learning scenario based upon a recent framework of self-supervised visual representation learning. We first observe that the out-of-class samples in the open-set unlabeled dataset can be identified effectively via self-supervised contrastive learning. Then, OpenCoS utilizes this information to overcome the failure modes in the existing state-of-the-art semi-supervised methods, by utilizing one-hot pseudo-labels and soft-labels for the identified in- and out-of-class unlabeled data, respectively. Our extensive experimental results show the effectiveness of OpenCoS under the presence of out-of-class samples, fixing up the state-of-the-art semi-supervised methods to be suitable for diverse scenarios involving open-set unlabeled data.
arXiv commentECCV Workshop on Learning from Limited and Imperfect Data, 2022. Code is available at https://github.com/alinlab/OpenCoS
References in corpus (17)
- A Simple Framework for Contrastive Learning of Visual Representations
- mixup: Beyond Empirical Risk Minimization
- Bootstrap your own latent: A new approach to self-supervised Learning
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Improved Baselines with Momentum Contrastive Learning
- Wide Residual Networks
- Unsupervised Data Augmentation for Consistency Training
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
- Robustness May Be at Odds with Accuracy
- Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
- Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
- Adversarial Examples Improve Image Recognition
- RealMix: Towards Realistic Semi-Supervised Deep Learning Algorithms