Negative sampling in semi-supervised learning
arXiv:1911.05166
Abstract
We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS3L loss to state-of-the-art SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the NS3L loss to MixMatch, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla MixMatch. We conduct extensive experiments on the CIFAR10, CIFAR100, SVHN and STL10 benchmark datasets.
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Distributed Representations of Words and Phrases and their Compositionality
- mixup: Beyond Empirical Risk Minimization
- Wide Residual Networks
- Unsupervised Data Augmentation for Consistency Training
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Interpolation Consistency Training for Semi-Supervised Learning
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- Semi-Supervised Learning with Generative Adversarial Networks
- Variational Autoencoder for Deep Learning of Images, Labels and Captions
- Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
- Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
- Generalization error bounds in semi-supervised classification under the cluster assumption