Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training

Abstract

Self-training is a standard approach to semi-supervised learning where the learner's own predictions on unlabeled data are used as supervision during training. In this paper, we reinterpret this label assignment process as an optimal transportation problem between examples and classes, wherein the cost of assigning an example to a class is mediated by the current predictions of the classifier. This formulation facilitates a practical annealing strategy for label assignment and allows for the inclusion of prior knowledge on class proportions via flexible upper bound constraints. The solutions to these assignment problems can be efficiently approximated using Sinkhorn iteration, thus enabling their use in the inner loop of standard stochastic optimization algorithms. We demonstrate the effectiveness of our algorithm on the CIFAR-10, CIFAR-100, and SVHN datasets in comparison with FixMatch, a state-of-the-art self-training algorithm. Our code is available at https://github.com/stanford-futuredata/sinkhorn-label-allocation.

arXiv commentICML 2021 camera ready version

References in corpus (10)

• Improved Regularization of Convolutional Neural Networks with Cutout
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• On Calibration of Modern Neural Networks
• Unsupervised Data Augmentation for Consistency Training
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Learning with Pseudo-Ensembles
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• Regularized Learning for Domain Adaptation under Label Shifts
• Deep multi-class learning from label proportions