AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
arXiv:2011.11779
Abstract
Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data. We propose AlphaMatch, an efficient SSL method that leverages data augmentations, by efficiently enforcing the label consistency between the data points and the augmented data derived from them. Our key technical contribution lies on: 1) using alpha-divergence to prioritize the regularization on data with high confidence, achieving a similar effect as FixMatch but in a more flexible fashion, and 2) proposing an optimization-based, EM-like algorithm to enforce the consistency, which enjoys better convergence than iterative regularization procedures used in recent SSL methods such as FixMatch, UDA, and MixMatch. AlphaMatch is simple and easy to implement, and consistently outperforms prior arts on standard benchmarks, e.g. CIFAR-10, SVHN, CIFAR-100, STL-10. Specifically, we achieve 91.3% test accuracy on CIFAR-10 with just 4 labelled data per class, substantially improving over the previously best 88.7% accuracy achieved by FixMatch.
preprint
References in corpus (13)
- Improved Regularization of Convolutional Neural Networks with Cutout
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Unsupervised Data Augmentation for Consistency Training
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Temporal Ensembling for Semi-Supervised Learning
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Semi-Supervised Learning with Ladder Networks
- Exploring Simple Siamese Representation Learning
- Self-Training for End-to-End Speech Recognition
- ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
- Adversarially Robust Generalization Just Requires More Unlabeled Data
- Learning by Transduction
- AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling