Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning
arXiv:2007.01293
Abstract
Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for every unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.
arXiv commentNeurIPS camera ready
References in corpus (9)
- Wide Residual Networks
- Unsupervised Data Augmentation for Consistency Training
- Understanding Black-box Predictions via Influence Functions
- Learning to Reweight Examples for Robust Deep Learning
- Semi-supervised Sequence Learning
- Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
- On the Accuracy of Influence Functions for Measuring Group Effects
- Efficient Structured Prediction with Latent Variables for General Graphical Models
- Auto-Vectorizing TensorFlow Graphs: Jacobians, Auto-Batching And Beyond