Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

Abstract

The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks. It maintains an exponential moving average of label predictions on each training example, and penalizes predictions that are inconsistent with this target. However, because the targets change only once per epoch, Temporal Ensembling becomes unwieldy when learning large datasets. To overcome this problem, we propose Mean Teacher, a method that averages model weights instead of label predictions. As an additional benefit, Mean Teacher improves test accuracy and enables training with fewer labels than Temporal Ensembling. Without changing the network architecture, Mean Teacher achieves an error rate of 4.35% on SVHN with 250 labels, outperforming Temporal Ensembling trained with 1000 labels. We also show that a good network architecture is crucial to performance. Combining Mean Teacher and Residual Networks, we improve the state of the art on CIFAR-10 with 4000 labels from 10.55% to 6.28%, and on ImageNet 2012 with 10% of the labels from 35.24% to 9.11%.

In this version: Corrected hyperparameters of the 4000-label CIFAR-10 ResNet experiment. Changed Antti's contact info, Advances in Neural Information Processing Systems 30 (NIPS 2017) pre-proceedings

Cited by in corpus (497)

• Knowledge Distillation: A Survey
• Bootstrap your own latent: A new approach to self-supervised Learning
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Unsupervised Data Augmentation for Consistency Training
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Recent advances and clinical applications of deep learning in medical image analysis
• Interpolation Consistency Training for Semi-Supervised Learning
• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
• AutoAugment: Learning Augmentation Policies from Data
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images
• A DIRT-T Approach to Unsupervised Domain Adaptation
• The Emerging Trends of Multi-Label Learning
• Rethinking Pre-training and Self-training
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• A Simple Semi-Supervised Learning Framework for Object Detection
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• Sound Event Detection: A Tutorial
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• An Overview of Deep Semi-Supervised Learning
• Self-training with Noisy Student improves ImageNet classification
• Rethinking the Value of Labels for Improving Class-Imbalanced Learning
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation
• Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
• DMT: Dynamic Mutual Training for Semi-Supervised Learning
• SPICE: Semantic Pseudo-labeling for Image Clustering
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Semi-supervised semantic segmentation needs strong, varied perturbations
• AutoNovel: Automatically Discovering and Learning Novel Visual Categories
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Self-Training: A Survey
• A Survey of Label-noise Representation Learning: Past, Present and Future
• Hybrid Dynamic Contrast and Probability Distillation for Unsupervised Person Re-Id
• SelfCF: A Simple Framework for Self-supervised Collaborative Filtering
• Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data
• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning
• Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
• Self-supervised Autoregressive Domain Adaptation for Time Series Data
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• Applications and Techniques for Fast Machine Learning in Science
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
• Robust Self-Ensembling Network for Hyperspectral Image Classification
• Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation
• Unlabeled Data Improves Adversarial Robustness
• Self-paced and self-consistent co-training for semi-supervised image segmentation
• Bootstrapping Semantic Segmentation with Regional Contrast
• Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
• Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation
• Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
• Cycle Self-Training for Domain Adaptation
• TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation
• FDA: Fourier Domain Adaptation for Semantic Segmentation
• Self-ensembling for visual domain adaptation
• A Survey of Deep Learning for Scientific Discovery
• Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
• A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
• SELF: Learning to Filter Noisy Labels with Self-Ensembling
• Provably Consistent Partial-Label Learning
• Uncertainty Aware Semi-Supervised Learning on Graph Data
• Self-Distillation as Instance-Specific Label Smoothing
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Transferrable Prototypical Networks for Unsupervised Domain Adaptation
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Data-Efficient Reinforcement Learning with Self-Predictive Representations
• Relational Knowledge Distillation
• Batch Virtual Adversarial Training for Graph Convolutional Networks
• Structured Consistency Loss for semi-supervised semantic segmentation
• Synergic Adversarial Label Learning for Grading Retinal Diseases via Knowledge Distillation and Multi-task Learning
• Denoising Multi-Source Weak Supervision for Neural Text Classification
• Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance
• Learning Target Domain Specific Classifier for Partial Domain Adaptation
• ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning
• Enhancing Diversity in Teacher-Student Networks via Asymmetric branches for Unsupervised Person Re-identification
• MT-UDA: Towards Unsupervised Cross-modality Medical Image Segmentation with Limited Source Labels
• SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
• Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning
• Class-Imbalanced Semi-Supervised Learning
• Neural Data Augmentation via Example Extrapolation
• Hierarchical Consistency Regularized Mean Teacher for Semi-supervised 3D Left Atrium Segmentation
• SenseCare: A Research Platform for Medical Image Informatics and Interactive 3D Visualization
• Confidence Scores Make Instance-dependent Label-noise Learning Possible
• High-Quality Self-Supervised Deep Image Denoising
• Automatically Discovering and Learning New Visual Categories with Ranking Statistics
• CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
• Confidence Regularized Self-Training
• Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs
• Pretraining Representations for Data-Efficient Reinforcement Learning
• Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation
• Regularizing Class-wise Predictions via Self-knowledge Distillation
• Exploring Object Relation in Mean Teacher for Cross-Domain Detection
• Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation
• Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels
• Source Free Domain Adaptation with Image Translation
• Supervised Domain Adaptation: A Graph Embedding Perspective and a Rectified Experimental Protocol
• Meta-Learned Confidence for Few-shot Learning
• Unbiased Teacher for Semi-Supervised Object Detection
• Parting with Illusions about Deep Active Learning
• CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
• SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning
• Decoupled Adaptation for Cross-Domain Object Detection
• Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework
• The Sound Demixing Challenge 2023 $\unicode{x2013}$ Music Demixing Track
• Smooth Neighbors on Teacher Graphs for Semi-supervised Learning
• Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations
• Revisiting Locally Supervised Learning: an Alternative to End-to-end Training
• Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency
• GradAug: A New Regularization Method for Deep Neural Networks
• SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling
• Regularizing Deep Networks with Semantic Data Augmentation
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• FEED: Feature-level Ensemble for Knowledge Distillation
• Quantitative Phase Imaging and Artificial Intelligence: A Review
• Learning to Discover Novel Visual Categories via Deep Transfer Clustering
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• Distilling Knowledge from Graph Convolutional Networks
• Cluster Alignment with a Teacher for Unsupervised Domain Adaptation
• Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptation
• Unsupervised Semantic Aggregation and Deformable Template Matching for Semi-Supervised Learning
• ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation
• BYOL works even without batch statistics
• Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification
• SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud
• Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation
• Unbiased Mean Teacher for Cross-domain Object Detection
• Contrastive Syn-to-Real Generalization
• Multiple Expert Brainstorming for Domain Adaptive Person Re-identification
• Combining MixMatch and Active Learning for Better Accuracy with Fewer Labels
• Multi-class Classification without Multi-class Labels
• RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning
• DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort
• Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
• Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
• On Data-Augmentation and Consistency-Based Semi-Supervised Learning
• When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets
• Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation
• Adapted Center and Scale Prediction: More Stable and More Accurate
• Semi-supervised learning based on generative adversarial network: a comparison between good GAN and bad GAN approach
• EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformations
• Guided Collaborative Training for Pixel-wise Semi-Supervised Learning
• Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners
• Semi-Supervised Domain Generalization with Stochastic StyleMatch
• Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation
• Towards Noise-resistant Object Detection with Noisy Annotations
• Semi-supervised classification of radiology images with NoTeacher: A Teacher that is not Mean
• FocalMix: Semi-Supervised Learning for 3D Medical Image Detection
• Cross-Referencing Self-Training Network for Sound Event Detection in Audio Mixtures
• Every Annotation Counts: Multi-label Deep Supervision for Medical Image Segmentation
• Auxiliary Task Reweighting for Minimum-data Learning
• Detection and Tracking Meet Drones Challenge
• Manifold regularization with GANs for semi-supervised learning
• FedCon: A Contrastive Framework for Federated Semi-Supervised Learning
• SimPLE: Similar Pseudo Label Exploitation for Semi-Supervised Classification
• Monocular 3D Detection with Geometric Constraints Embedding and Semi-supervised Training
• Energy Models for Better Pseudo-Labels: Improving Semi-Supervised Classification with the 1-Laplacian Graph Energy
• Self-supervised Temporal Discriminative Learning for Video Representation Learning
• Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations
• Multiple Domain Experts Collaborative Learning: Multi-Source Domain Generalization For Person Re-Identification
• Interpolation-based semi-supervised learning for object detection
• Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching
• Heavily Augmented Sound Event Detection utilizing Weak Predictions
• Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning
• Semi-supervised Active Learning for Instance Segmentation via Scoring Predictions
• Deep Metric Transfer for Label Propagation with Limited Annotated Data
• ICE: Inter-instance Contrastive Encoding for Unsupervised Person Re-identification
• SESS: Self-Ensembling Semi-Supervised 3D Object Detection
• RealMix: Towards Realistic Semi-Supervised Deep Learning Algorithms
• CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation
• Test time Adaptation through Perturbation Robustness
• Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation
• Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model
• Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation
• Proposal Learning for Semi-Supervised Object Detection
• Self-Ensembling Contrastive Learning for Semi-Supervised Medical Image Segmentation
• A Fourier-based Framework for Domain Generalization
• Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representations
• Momentum^2 Teacher: Momentum Teacher with Momentum Statistics for Self-Supervised Learning
• CO2: Consistent Contrast for Unsupervised Visual Representation Learning
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
• Iterative Graph Self-Distillation
• Training Sound Event Detection On A Heterogeneous Dataset
• FedSiam: Towards Adaptive Federated Semi-Supervised Learning
• 3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection
• SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning
• Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach
• Multimodal Semi-Supervised Learning for 3D Objects
• Adaptive Hierarchical Dual Consistency for Semi-Supervised Left Atrium Segmentation on Cross-Domain Data
• Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning
• MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering
• Self-training with noisy student model and semi-supervised loss function for dcase 2021 challenge task 4
• Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift
• Learning from Temporal Gradient for Semi-supervised Action Recognition
• Federated Semi-Supervised Learning with Class Distribution Mismatch
• Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification
• Triple Generative Adversarial Networks
• Self-supervised Mean Teacher for Semi-supervised Chest X-ray Classification
• Dual-Task Mutual Learning for Semi-Supervised Medical Image Segmentation
• FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning
• Boosting Light-Weight Depth Estimation Via Knowledge Distillation
• Self-Supervised Learning for Gastritis Detection with Gastric X-ray Images
• Semi-Supervised Domain Generalizable Person Re-Identification
• Selective Distillation of Weakly Annotated GTD for Vision-based Slab Identification System
• Instance Credibility Inference for Few-Shot Learning
• Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning
• Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
• Progressive Cross-camera Soft-label Learning for Semi-supervised Person Re-identification
• Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
• Circumventing Outliers of AutoAugment with Knowledge Distillation
• Semi-Supervised Medical Image Segmentation with Co-Distribution Alignment
• Learning to Impute: A General Framework for Semi-supervised Learning
• From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation
• Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
• The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
• Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection
• DINE: Domain Adaptation from Single and Multiple Black-box Predictors
• TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning
• Learning from Label Proportions with Consistency Regularization
• Revisiting consistency for semi-supervised semantic segmentation
• Semi-Supervised Learning with Normalizing Flows
• Deep Semi-Supervised Learning for Time Series Classification
• A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model
• Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• A Three-Stage Self-Training Framework for Semi-Supervised Semantic Segmentation
• Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification
• Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation
• Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing
• Confidence Adaptive Regularization for Deep Learning with Noisy Labels
• Semi-supervised Viewpoint Estimation with Geometry-aware Conditional Generation
• Local Label Propagation for Large-Scale Semi-Supervised Learning
• An analysis of over-sampling labeled data in semi-supervised learning with FixMatch
• VideoSSL: Semi-Supervised Learning for Video Classification
• Deep learning for Chemometric and non-translational data
• Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning
• Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognition
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels
• Knowledge Distillation with Adaptive Asymmetric Label Sharpening for Semi-supervised Fracture Detection in Chest X-rays
• Semi-supervised Image Classification with Grad-CAM Consistency
• Robust Semantic Segmentation with Superpixel-Mix
• Adaptive Early-Learning Correction for Segmentation from Noisy Annotations
• What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels
• Density-Aware Graph for Deep Semi-Supervised Visual Recognition
• MetaMixUp: Learning Adaptive Interpolation Policy of MixUp with Meta-Learning
• Countering Noisy Labels By Learning From Auxiliary Clean Labels
• Every Node Counts: Self-Ensembling Graph Convolutional Networks for Semi-Supervised Learning
• Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation
• Robust Training of Social Media Image Classification Models for Rapid Disaster Response
• FedSEAL: Semi-Supervised Federated Learning with Self-Ensemble Learning and Negative Learning
• Semi-Supervised Hypothesis Transfer for Source-Free Domain Adaptation
• Learning to Learn in a Semi-Supervised Fashion
• Data-Uncertainty Guided Multi-Phase Learning for Semi-Supervised Object Detection
• Attribute-aware Pedestrian Detection in a Crowd
• DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
• Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis
• Semi-Supervised Crowd Counting via Self-Training on Surrogate Tasks
• Edge Bias in Federated Learning and its Solution by Buffered Knowledge Distillation
• SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations
• Negative sampling in semi-supervised learning
• Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach
• Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation
• GAN Cocktail: mixing GANs without dataset access
• Multiple Fusion Adaptation: A Strong Framework for Unsupervised Semantic Segmentation Adaptation
• Repetitive Reprediction Deep Decipher for Semi-Supervised Learning
• Flow Contrastive Estimation of Energy-Based Models
• Semi-supervised Keypoint Localization
• Online Knowledge Distillation via Multi-branch Diversity Enhancement
• CORE-ReID V2: Advancing the Domain Adaptation for Object Re-Identification with Optimized Training and Ensemble Fusion
• 3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning
• 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos
• Improving Contrastive Learning by Visualizing Feature Transformation
• Noisy Labels are Treasure: Mean-Teacher-Assisted Confident Learning for Hepatic Vessel Segmentation
• Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition
• More unlabelled data or label more data? A study on semi-supervised laparoscopic image segmentation
• Pseudo-labeling for Scalable 3D Object Detection
• Manifold Graph with Learned Prototypes for Semi-Supervised Image Classification
• Built Year Prediction from Buddha Face with Heterogeneous Labels
• A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation
• MixPUL: Consistency-based Augmentation for Positive and Unlabeled Learning
• DualDis: Dual-Branch Disentangling with Adversarial Learning
• Boosting Active Learning for Speech Recognition with Noisy Pseudo-labeled Samples
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding
• ImageNet-21K Pretraining for the Masses
• Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction
• Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images
• Disentanglement-based Cross-Domain Feature Augmentation for Effective Unsupervised Domain Adaptive Person Re-identification
• One-bit Supervision for Image Classification
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation
• The Mathematical Foundations of Manifold Learning
• StackMix: A complementary Mix algorithm
• Semi-supervised Learning using Adversarial Training with Good and Bad Samples
• Semi-supervised learning by selective training with pseudo labels via confidence estimation
• Pointwise Binary Classification with Pairwise Confidence Comparisons
• How Out-of-Distribution Data Hurts Semi-Supervised Learning
• Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones
• Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning
• Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
• Weakly Labeled Sound Event Detection Using Tri-training and Adversarial Learning
• A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
• Gotta Adapt 'Em All: Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild
• CUDA: Contradistinguisher for Unsupervised Domain Adaptation
• Semi-Supervised Crowd Counting from Unlabeled Data
• Grasping Detection Network with Uncertainty Estimation for Confidence-Driven Semi-Supervised Domain Adaptation
• Geographical Knowledge-driven Representation Learning for Remote Sensing Images
• Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data
• HODGEPODGE: Sound event detection based on ensemble of semi-supervised learning methods
• Unsupervised Domain Adaptation using Generative Models and Self-ensembling
• Lightweight and Robust Representation of Economic Scales from Satellite Imagery
• Multi-Anchor Active Domain Adaptation for Semantic Segmentation
• Learning a Domain Classifier Bank for Unsupervised Adaptive Object Detection
• Where is the Bottleneck of Adversarial Learning with Unlabeled Data?
• Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information
• Exploiting Contextual Information with Deep Neural Networks
• Graph Consistency Based Mean-Teaching for Unsupervised Domain Adaptive Person Re-Identification
• Semi-Supervised Learning for Fetal Brain MRI Quality Assessment with ROI consistency
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
• Adversarial Transformations for Semi-Supervised Learning
• CADA: Multi-scale Collaborative Adversarial Domain Adaptation for Unsupervised Optic Disc and Cup Segmentation
• Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
• Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation
• SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation
• Unsupervised Domain Generalization for Person Re-identification: A Domain-specific Adaptive Framework
• Automatic CT Segmentation from Bounding Box Annotations using Convolutional Neural Networks
• Evaluating Self and Semi-Supervised Methods for Remote Sensing Segmentation Tasks
• Semi-supervised Semantic Segmentation with Directional Context-aware Consistency
• Deep Recurrent Semi-Supervised EEG Representation Learning for Emotion Recognition
• Multilevel Knowledge Transfer for Cross-Domain Object Detection
• Reducing Spatial Labeling Redundancy for Semi-supervised Crowd Counting
• Progressive Representative Labeling for Deep Semi-Supervised Learning
• Semi-supervised Learning for Identifying the Likelihood of Agitation in People with Dementia
• Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
• Consistency Regularization Can Improve Robustness to Label Noise
• Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data
• Self-supervised Change Detection in Multi-view Remote Sensing Images
• Semi-supervised Domain Adaptation for Semantic Segmentation
• Exploiting Cross-Modal Prediction and Relation Consistency for Semi-Supervised Image Captioning
• Land Cover Mapping in Limited Labels Scenario: A Survey
• MUSCLE: Strengthening Semi-Supervised Learning Via Concurrent Unsupervised Learning Using Mutual Information Maximization
• Ranking Neural Checkpoints
• Semi-supervised Relation Extraction via Incremental Meta Self-Training
• Adversarial Imitation Learning with Trajectorial Augmentation and Correction
• How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning
• A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning
• Learning to Detect Important People in Unlabelled Images for Semi-supervised Important People Detection
• Teaching Machine Comprehension with Compositional Explanations
• Optimally Combining Classifiers for Semi-Supervised Learning
• Exploring Category-Agnostic Clusters for Open-Set Domain Adaptation
• Local Clustering with Mean Teacher for Semi-supervised Learning
• Target Consistency for Domain Adaptation: when Robustness meets Transferability
• Semi-supervised Sequential Generative Models
• Temporal Self-Ensembling Teacher for Semi-Supervised Object Detection
• Self-similarity Student for Partial Label Histopathology Image Segmentation
• Training Object Detectors from Few Weakly-Labeled and Many Unlabeled Images
• Renal Cell Carcinoma Detection and Subtyping with Minimal Point-Based Annotation in Whole-Slide Images
• Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets
• Adaptive Distillation for Decentralized Learning from Heterogeneous Clients
• Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptation
• Extending Label Smoothing Regularization with Self-Knowledge Distillation
• NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction
• Adversarial-Based Knowledge Distillation for Multi-Model Ensemble and Noisy Data Refinement
• Semi-Supervised Learning for In-Game Expert-Level Music-to-Dance Translation
• Mixup-breakdown: a consistency training method for improving generalization of speech separation models
• Non-technical Loss Detection with Statistical Profile Images Based on Semi-supervised Learning
• SSAH: Semi-supervised Adversarial Deep Hashing with Self-paced Hard Sample Generation
• A Comprehensive Approach to Unsupervised Embedding Learning based on AND Algorithm
• Domain Agnostic Real-Valued Specificity Prediction
• Reliable Semi-Supervised Learning when Labels are Missing at Random
• Label Prediction Framework for Semi-Supervised Cross-Modal Retrieval
• Certainty Driven Consistency Loss on Multi-Teacher Networks for Semi-Supervised Learning
• Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images
• Active Learning for Sound Event Detection
• X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
• Modular Adaptation for Cross-Domain Few-Shot Learning
• Data Summarization via Bilevel Optimization
• Deep Ensemble Collaborative Learning by using Knowledge-transfer Graph for Fine-grained Object Classification
• Contrastive Mixup: Self- and Semi-Supervised learning for Tabular Domain
• Interpretable Foreground Object Search As Knowledge Distillation
• Hierarchical Deep Network with Uncertainty-aware Semi-supervised Learning for Vessel Segmentation
• Improving Object Detection with Selective Self-supervised Self-training
• Learning Temporal Action Proposals With Fewer Labels
• Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification
• Non-Negative Matrix Factorization-Convolutional Neural Network (NMF-CNN) For Sound Event Detection
• Guided learning for weakly-labeled semi-supervised sound event detection
• Transform consistency for learning with noisy labels
• CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
• Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining
• A New Distribution on the Simplex with Auto-Encoding Applications
• Omni-supervised Facial Expression Recognition via Distilled Data
• Temporal Knowledge Consistency for Unsupervised Visual Representation Learning
• Matching Distributions via Optimal Transport for Semi-Supervised Learning
• Adversarial Dropout for Recurrent Neural Networks
• AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
• Teacher-Student Asynchronous Learning with Multi-Source Consistency for Facial Landmark Detection
• Multi-level Knowledge Distillation via Knowledge Alignment and Correlation
• Out-distribution aware Self-training in an Open World Setting
• Weakly-Supervised Dense Action Anticipation
• Learning with Multiplicative Perturbations
• Self-supervised Learning for Semi-supervised Temporal Language Grounding
• Patch-level Neighborhood Interpolation: A General and Effective Graph-based Regularization Strategy
• Power Pooling Operators and Confidence Learning for Semi-Supervised Sound Event Detection
• Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection
• Attribute-Induced Bias Eliminating for Transductive Zero-Shot Learning
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Scribble-Supervised Semantic Segmentation by Random Walk on Neural Representation and Self-Supervision on Neural Eigenspace
• Semi-supervised Learning for COVID-19 Image Classification via ResNet
• S4-Net: Geometry-Consistent Semi-Supervised Semantic Segmentation
• Semixup: In- and Out-of-Manifold Regularization for Deep Semi-Supervised Knee Osteoarthritis Severity Grading from Plain Radiographs
• A Review on Semi-Supervised Relation Extraction
• Ensemble Learning with Manifold-Based Data Splitting for Noisy Label Correction
• Consistent Posterior Distributions under Vessel-Mixing: A Regularization for Cross-Domain Retinal Artery/Vein Classification
• From Consensus to Disagreement: Multi-Teacher Distillation for Semi-Supervised Relation Extraction
• Multi-Purposing Domain Adaptation Discriminators for Pseudo Labeling Confidence
• Reliable Weakly Supervised Learning: Maximize Gain and Maintain Safeness
• Multimodal Knowledge Expansion
• Deep Momentum Uncertainty Hashing
• Dynamic Slimmable Denoising Network
• Dynamic Slimmable Network
• MisMatch: Calibrated Segmentation via Consistency on Differential Morphological Feature Perturbations with Limited Labels
• Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift
• RCT: Random Consistency Training for Semi-supervised Sound Event Detection
• CFEA: Collaborative Feature Ensembling Adaptation for Domain Adaptation in Unsupervised Optic Disc and Cup Segmentation
• Semi-supervised Learning for Dense Object Detection in Retail Scenes
• Unknown Identity Rejection Loss: Utilizing Unlabeled Data for Face Recognition
• LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching
• FROST: Faster and more Robust One-shot Semi-supervised Training
• OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning
• Regularization with Latent Space Virtual Adversarial Training
• Semi-Supervised Domain Generalization with Evolving Intermediate Domain
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation
• Self-Tuning for Data-Efficient Deep Learning
• An empirical study of domain-agnostic semi-supervised learning via energy-based models: joint-training and pre-training
• Teacher-Student Competition for Unsupervised Domain Adaptation
• Semi-supervised Sound Event Detection using Random Augmentation and Consistency Regularization
• Semi-Supervised Learning for Bone Mineral Density Estimation in Hip X-ray Images
• SSLayout360: Semi-Supervised Indoor Layout Estimation from 360-Degree Panorama
• Iterative Self-Learning: Semi-Supervised Improvement to Dataset Volumes and Model Accuracy
• Layer-Wise Multi-View Learning for Neural Machine Translation
• Soft-Median Choice: An Automatic Feature Smoothing Method for Sound Event Detection
• Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
• Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation
• A Survey on Green Deep Learning
• Semi-Supervised Learning with Meta-Gradient
• OXnet: Omni-supervised Thoracic Disease Detection from Chest X-rays
• Semi-supervised learning method based on predefined evenly-distributed class centroids
• Semi-Supervised Text Classification via Self-Pretraining
• Robust Temporal Ensembling for Learning with Noisy Labels
• Learning Metrics from Mean Teacher: A Supervised Learning Method for Improving the Generalization of Speaker Verification System
• Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction
• A Self-ensembling Framework for Semi-supervised Knee Cartilage Defects Assessment with Dual-Consistency
• ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Medical Image Segmentation
• Scalable Semi-supervised Landmark Localization for X-ray Images using Few-shot Deep Adaptive Graph
• Domain Constraint Approximation based Semi Supervision
• FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks
• Cross-Modality Domain Adaptation for Vestibular Schwannoma and Cochlea Segmentation
• Multilingual and Multilabel Emotion Recognition using Virtual Adversarial Training
• Pairwise Teacher-Student Network for Semi-Supervised Hashing
• Learning the Prediction Distribution for Semi-Supervised Learning with Normalising Flows
• Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filtering
• Pseudo Labeling and Negative Feedback Learning for Large-scale Multi-label Domain Classification
• Robust Ensembling Network for Unsupervised Domain Adaptation
• Boosting Unconstrained Face Recognition with Auxiliary Unlabeled Data
• Overcoming label noise in audio event detection using sequential labeling
• Dual-Teacher: Integrating Intra-domain and Inter-domain Teachers for Annotation-efficient Cardiac Segmentation
• Negative Pseudo Labeling using Class Proportion for Semantic Segmentation in Pathology
• Learn to Propagate Reliably on Noisy Affinity Graphs
• MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation
• VeniBot: Towards Autonomous Venipuncture with Semi-supervised Vein Segmentation from Ultrasound Images
• CycleCluster: Modernising Clustering Regularisation for Deep Semi-Supervised Classification
• Bootstrapping User and Item Representations for One-Class Collaborative Filtering
• Self-supervised learning using consistency regularization of spatio-temporal data augmentation for action recognition
• Learning Dual Retrieval Module for Semi-supervised Relation Extraction
• Audiovisual transfer learning for audio tagging and sound event detection
• Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering
• Skeptical Deep Learning with Distribution Correction
• Implicit Regularization of Bregman Proximal Point Algorithm and Mirror Descent on Separable Data
• Investigating the Effect of Intraclass Variability in Temporal Ensembling
• On the Efficiency of Subclass Knowledge Distillation in Classification Tasks
• Mutual Teaching for Graph Convolutional Networks
• Self Training with Ensemble of Teacher Models
• BiSTF: Bilateral-Branch Self-Training Framework for Semi-Supervised Large-scale Fine-Grained Recognition
• Snowball: Iterative Model Evolution and Confident Sample Discovery for Semi-Supervised Learning on Very Small Labeled Datasets
• Semi-supervised Acoustic Event Detection based on tri-training
• ActiveMatch: End-to-end Semi-supervised Active Representation Learning
• Hetero-Modal Learning and Expansive Consistency Constraints for Semi-Supervised Detection from Multi-Sequence Data
• Semi-supervised and Population Based Training for Voice Commands Recognition
• Domain-Specific Bias Filtering for Single Labeled Domain Generalization
• Semi-supervised Learning with Contrastive Predicative Coding