RankingMatch: Delving into Semi-Supervised Learning with Consistency Regularization and Ranking Loss
arXiv:2110.04430
Abstract
Semi-supervised learning (SSL) has played an important role in leveraging unlabeled data when labeled data is limited. One of the most successful SSL approaches is based on consistency regularization, which encourages the model to produce unchanged with perturbed input. However, there has been less attention spent on inputs that have the same label. Motivated by the observation that the inputs having the same label should have the similar model outputs, we propose a novel method, RankingMatch, that considers not only the perturbed inputs but also the similarity among the inputs having the same label. We especially introduce a new objective function, dubbed BatchMean Triplet loss, which has the advantage of computational efficiency while taking into account all input samples. Our RankingMatch achieves state-of-the-art performance across many standard SSL benchmarks with a variety of labeled data amounts, including 95.13% accuracy on CIFAR-10 with 250 labels, 77.65% accuracy on CIFAR-100 with 10000 labels, 97.76% accuracy on SVHN with 250 labels, and 97.77% accuracy on SVHN with 1000 labels. We also perform an ablation study to prove the efficacy of the proposed BatchMean Triplet loss against existing versions of Triplet loss.
References in corpus (12)
- A Simple Framework for Contrastive Learning of Visual Representations
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- In Defense of the Triplet Loss for Person Re-Identification
- Improved Regularization of Convolutional Neural Networks with Cutout
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Wide Residual Networks
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Temporal Ensembling for Semi-Supervised Learning
- Deep Learning Scaling is Predictable, Empirically
- Stacked What-Where Auto-encoders
- RealMix: Towards Realistic Semi-Supervised Deep Learning Algorithms
- Learning to Learn in a Semi-Supervised Fashion