OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers
arXiv:2105.14148
Abstract
Semi-supervised learning (SSL) is an effective means to leverage unlabeled data to improve a model's performance. Typical SSL methods like FixMatch assume that labeled and unlabeled data share the same label space. However, in practice, unlabeled data can contain categories unseen in the labeled set, i.e., outliers, which can significantly harm the performance of SSL algorithms. To address this problem, we propose a novel Open-set Semi-Supervised Learning (OSSL) approach called OpenMatch. Learning representations of inliers while rejecting outliers is essential for the success of OSSL. To this end, OpenMatch unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers. The OVA-classifier outputs the confidence score of a sample being an inlier, providing a threshold to detect outliers. Another key contribution is an open-set soft-consistency regularization loss, which enhances the smoothness of the OVA-classifier with respect to input transformations and greatly improves outlier detection. OpenMatch achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
arXiv commentCode https://github.com/VisionLearningGroup/OP_Match
References in corpus (19)
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Wide Residual Networks
- Unsupervised Data Augmentation for Consistency Training
- Temporal Ensembling for Semi-Supervised Learning
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- Recent Advances in Open Set Recognition: A Survey
- Interpolation Consistency Training for Semi-Supervised Learning
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- Deep Anomaly Detection with Outlier Exposure
- Learning Confidence for Out-of-Distribution Detection in Neural Networks
- Distributional Smoothing with Virtual Adversarial Training
- CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
- ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
- Contrastive Training for Improved Out-of-Distribution Detection
- On Data-Augmentation and Consistency-Based Semi-Supervised Learning
- OVANet: One-vs-All Network for Universal Domain Adaptation
- Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks