An Overview of Deep Semi-Supervised Learning

Abstract

Deep neural networks demonstrated their ability to provide remarkable performances on a wide range of supervised learning tasks (e.g., image classification) when trained on extensive collections of labeled data (e.g., ImageNet). However, creating such large datasets requires a considerable amount of resources, time, and effort. Such resources may not be available in many practical cases, limiting the adoption and the application of many deep learning methods. In a search for more data-efficient deep learning methods to overcome the need for large annotated datasets, there is a rising research interest in semi-supervised learning and its applications to deep neural networks to reduce the amount of labeled data required, by either developing novel methods or adopting existing semi-supervised learning frameworks for a deep learning setting. In this paper, we provide a comprehensive overview of deep semi-supervised learning, starting with an introduction to the field, followed by a summarization of the dominant semi-supervised approaches in deep learning.

Preprint

References in corpus (50)

• Distributed Representations of Words and Phrases and their Compositionality
• Auto-Encoding Variational Bayes
• Semi-Supervised Classification with Graph Convolutional Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• mixup: Beyond Empirical Risk Minimization
• Self-Attention Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Graph Embedding Techniques, Applications, and Performance: A Survey
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Semi-Supervised Learning with Deep Generative Models
• Unsupervised Representation Learning by Predicting Image Rotations
• Graph Neural Networks: A Review of Methods and Applications
• Tutorial on Variational Autoencoders
• Momentum Contrast for Unsupervised Visual Representation Learning
• Energy-based Generative Adversarial Network
• Variational Graph Auto-Encoders
• Adversarial Feature Learning
• Adversarially Learned Inference
• Dual Learning for Machine Translation
• Semi-Supervised Learning with Ladder Networks
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Semi-Supervised Learning with Generative Adversarial Networks
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Billion-scale semi-supervised learning for image classification
• A Gentle Introduction to Deep Learning for Graphs
• Auxiliary Deep Generative Models
• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
• Watch Your Step: Learning Node Embeddings via Graph Attention
• HARP: Hierarchical Representation Learning for Networks
• Co-regularized Alignment for Unsupervised Domain Adaptation
• Understanding Back-Translation at Scale
• Combating Label Noise in Deep Learning Using Abstention
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Co-training for Demographic Classification Using Deep Learning from Label Proportions
• Self-ensembling for visual domain adaptation
• Consistency Regularization for Generative Adversarial Networks
• Domain Adaptations for Computer Vision Applications
• Neural Simpletrons - Minimalistic Directed Generative Networks for Learning with Few Labels
• Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost
• Leveraging Just a Few Keywords for Fine-Grained Aspect Detection Through Weakly Supervised Co-Training

Cited by in corpus (15)

• A Survey on Deep Semi-supervised Learning
• Social physics
• Machine Learning in Aerodynamic Shape Optimization
• Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
• Review: Deep Learning in Electron Microscopy
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge
• The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics
• Semi-Supervised Hierarchical Graph Classification
• BenthicNet: A global compilation of seafloor images for deep learning applications
• A Comparative Review of Recent Few-Shot Object Detection Algorithms
• A Comprehensive Study on Medical Image Segmentation using Deep Neural Networks
• A Brief Overview of Unsupervised Neural Speech Representation Learning
• Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler
• Minimizing Labeling Effort for Tree Skeleton Segmentation using an Automated Iterative Training Methodology