Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning

Abstract

While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced. However, these SSL algorithms trained under imbalanced class distributions can severely suffer when generalizing to a balanced testing criterion, since they utilize biased pseudo-labels of unlabeled data toward majority classes. To alleviate this issue, we formulate a convex optimization problem to softly refine the pseudo-labels generated from the biased model, and develop a simple algorithm, named Distribution Aligning Refinery of Pseudo-label (DARP) that solves it provably and efficiently. Under various class-imbalanced semi-supervised scenarios, we demonstrate the effectiveness of DARP and its compatibility with state-of-the-art SSL schemes.

19 pages; NeurIPS 2020

References in corpus (12)

• SMOTE: Synthetic Minority Over-sampling Technique
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Wide Residual Networks
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Learning to Reweight Examples for Robust Deep Learning
• Self-training with Noisy Student improves ImageNet classification
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Regularized Learning for Domain Adaptation under Label Shifts
• RealMix: Towards Realistic Semi-Supervised Deep Learning Algorithms

Cited by in corpus (6)

• CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
• Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
• Class-Specific Distribution Alignment for Semi-Supervised Medical Image Classification
• Single-Modal Entropy based Active Learning for Visual Question Answering
• Addressing Class Variable Imbalance in Federated Semi-supervised Learning
• Long-Tailed Learning for Generalized Category Discovery