FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

Abstract

The recently proposed FixMatch achieved state-of-the-art results on most semi-supervised learning (SSL) benchmarks. However, like other modern SSL algorithms, FixMatch uses a pre-defined constant threshold for all classes to select unlabeled data that contribute to the training, thus failing to consider different learning status and learning difficulties of different classes. To address this issue, we propose Curriculum Pseudo Labeling (CPL), a curriculum learning approach to leverage unlabeled data according to the model's learning status. The core of CPL is to flexibly adjust thresholds for different classes at each time step to let pass informative unlabeled data and their pseudo labels. CPL does not introduce additional parameters or computations (forward or backward propagation). We apply CPL to FixMatch and call our improved algorithm FlexMatch. FlexMatch achieves state-of-the-art performance on a variety of SSL benchmarks, with especially strong performances when the labeled data are extremely limited or when the task is challenging. For example, FlexMatch achieves 13.96% and 18.96% error rate reduction over FixMatch on CIFAR-100 and STL-10 datasets respectively, when there are only 4 labels per class. CPL also significantly boosts the convergence speed, e.g., FlexMatch can use only 1/5 training time of FixMatch to achieve even better performance. Furthermore, we show that CPL can be easily adapted to other SSL algorithms and remarkably improve their performances. We open-source our code at https://github.com/TorchSSL/TorchSSL.

NeurIPS 2021; camera-ready version; 16 pages with appendix; code: https://github.com/TorchSSL/TorchSSL

References in corpus (1)

• Learning with Pseudo-Ensembles

Cited by in corpus (21)

• Memory-Guided Multi-View Multi-Domain Fake News Detection
• Self-Training: A Survey
• GALA: Graph Diffusion-based Alignment with Jigsaw for Source-free Domain Adaptation
• Federated Few-Shot Learning for Mobile NLP
• Versatile Teacher: A Class-aware Teacher-student Framework for Cross-domain Adaptation
• MLN-net: A multi-source medical image segmentation method for clustered microcalcifications using multiple layer normalization
• Exploiting Low-confidence Pseudo-labels for Source-free Object Detection
• RDA: Reciprocal Distribution Alignment for Robust Semi-supervised Learning
• Active Self-Semi-Supervised Learning for Few Labeled Samples
• Domain Adaptive Detection of MAVs: A Benchmark and Noise Suppression Network
• Class-Specific Distribution Alignment for Semi-Supervised Medical Image Classification
• Calibrated Adaptive Teacher for Domain Adaptive Intelligent Fault Diagnosis
• SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning
• Semi-Supervised Medical Image Segmentation with Co-Distribution Alignment
• Cycle Self-Training for Semi-Supervised Object Detection with Distribution Consistency Reweighting
• Reliable Student: Addressing Noise in Semi-Supervised 3D Object Detection
• BiPC: Bidirectional Probability Calibration for Unsupervised Domain Adaption
• Enhancing Dual Network Based Semi-Supervised Medical Image Segmentation with Uncertainty-Guided Pseudo-Labeling
• A Unified Contrastive Loss for Self-Training
• The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation
• High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling