CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning

Abstract

Semi-supervised learning on class-imbalanced data, although a realistic problem, has been under studied. While existing semi-supervised learning (SSL) methods are known to perform poorly on minority classes, we find that they still generate high precision pseudo-labels on minority classes. By exploiting this property, in this work, we propose Class-Rebalancing Self-Training (CReST), a simple yet effective framework to improve existing SSL methods on class-imbalanced data. CReST iteratively retrains a baseline SSL model with a labeled set expanded by adding pseudo-labeled samples from an unlabeled set, where pseudo-labeled samples from minority classes are selected more frequently according to an estimated class distribution. We also propose a progressive distribution alignment to adaptively adjust the rebalancing strength dubbed CReST+. We show that CReST and CReST+ improve state-of-the-art SSL algorithms on various class-imbalanced datasets and consistently outperform other popular rebalancing methods. Code has been made available at https://github.com/google-research/crest.

arXiv commentTo appear in CVPR 2021. Code release: https://github.com/google-research/crest

References in corpus (16)

• A systematic study of the class imbalance problem in convolutional neural networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Learning to Reweight Examples for Robust Deep Learning
• Semi-Supervised Learning with Ladder Networks
• What makes ImageNet good for transfer learning?
• Class-Balanced Loss Based on Effective Number of Samples
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning
• Large-Scale Long-Tailed Recognition in an Open World
• Long-tail learning via logit adjustment
• Class-Imbalanced Semi-Supervised Learning
• Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective
• Feature Transfer Learning for Deep Face Recognition with Under-Represented Data
• Equalization Loss for Long-Tailed Object Recognition

Cited by in corpus (6)

• ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning
• Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
• Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning
• Structured Sparse R-CNN for Direct Scene Graph Generation
• Addressing Class Variable Imbalance in Federated Semi-supervised Learning
• MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection