A Survey on Deep Semi-supervised Learning

Abstract

Deep semi-supervised learning is a fast-growing field with a range of practical applications. This paper provides a comprehensive survey on both fundamentals and recent advances in deep semi-supervised learning methods from perspectives of model design and unsupervised loss functions. We first present a taxonomy for deep semi-supervised learning that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods. Then we provide a comprehensive review of 52 representative methods and offer a detailed comparison of these methods in terms of the type of losses, contributions, and architecture differences. In addition to the progress in the past few years, we further discuss some shortcomings of existing methods and provide some tentative heuristic solutions for solving these open problems.

References in corpus (11)

• Distilling the Knowledge in a Neural Network
• In Defense of the Triplet Loss for Person Re-Identification
• Improved Regularization of Convolutional Neural Networks with Cutout
• Interpolation Consistency Training for Semi-Supervised Learning
• Semi-Supervised Learning with Generative Adversarial Networks
• Learning Graph Embedding with Adversarial Training Methods
• An Overview of Deep Semi-Supervised Learning
• GridMask Data Augmentation
• Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks
• Learning From Labeled And Unlabeled Data: An Empirical Study Across Techniques And Domains
• Hide-and-Seek: A Data Augmentation Technique for Weakly-Supervised Localization and Beyond

Cited by in corpus (20)

• Machine Learning in Aerodynamic Shape Optimization
• RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentation
• Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
• Self-Training: A Survey
• Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detection
• Deep Learning-based Techniques for Integrated Sensing and Communication Systems: State-of-the-Art, Challenges, and Opportunities
• TreeSketchNet: From Sketch To 3D Tree Parameters Generation
• Making Invisible Visible: Data-Driven Seismic Inversion with Spatio-temporally Constrained Data Augmentation
• BenthicNet: A global compilation of seafloor images for deep learning applications
• Class-Specific Distribution Alignment for Semi-Supervised Medical Image Classification
• Robust Semi-Supervised Learning for Self-learning Open-World Classes
• Label-Efficient Semantic Segmentation of LiDAR Point Clouds in Adverse Weather Conditions
• PEPL: Precision-Enhanced Pseudo-Labeling for Fine-Grained Image Classification in Semi-Supervised Learning
• Semi-Supervised Learning under General Causal Models
• SpecRegMatch: Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction
• Development and Adaptation of Robotic Vision in the Real-World: the Challenge of Door Detection
• Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning
• Link Polarity Prediction from Sparse and Noisy Labels via Multiscale Social Balance
• HACMatch Semi-Supervised Rotation Regression with Hardness-Aware Curriculum Pseudo Labeling
• A Point-Neighborhood Learning Framework for Nasal Endoscope Image Segmentation