Semi-Supervised Learning by Label Gradient Alignment
arXiv:1902.02336
Abstract
We present label gradient alignment, a novel algorithm for semi-supervised learning which imputes labels for the unlabeled data and trains on the imputed labels. We define a semantically meaningful distance metric on the input space by mapping a point (x, y) to the gradient of the model at (x, y). We then formulate an optimization problem whose objective is to minimize the distance between the labeled and the unlabeled data in this space, and we solve it by gradient descent on the imputed labels. We evaluate label gradient alignment using the standardized architecture introduced by Oliver et al. (2018) and demonstrate state-of-the-art accuracy in semi-supervised CIFAR-10 classification.
References in corpus (10)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Understanding Black-box Predictions via Influence Functions
- Semi-Supervised Learning with Ladder Networks
- Shake-Shake regularization
- There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
- Adapting Auxiliary Losses Using Gradient Similarity
- Learning by Association - A versatile semi-supervised training method for neural networks
- Semi-Supervised Learning via Compact Latent Space Clustering
- Gradient Similarity: An Explainable Approach to Detect Adversarial Attacks against Deep Learning