DMT: Dynamic Mutual Training for Semi-Supervised Learning

Abstract

Recent semi-supervised learning methods use pseudo supervision as core idea, especially self-training methods that generate pseudo labels. However, pseudo labels are unreliable. Self-training methods usually rely on single model prediction confidence to filter low-confidence pseudo labels, thus remaining high-confidence errors and wasting many low-confidence correct labels. In this paper, we point out it is difficult for a model to counter its own errors. Instead, leveraging inter-model disagreement between different models is a key to locate pseudo label errors. With this new viewpoint, we propose mutual training between two different models by a dynamically re-weighted loss function, called Dynamic Mutual Training (DMT). We quantify inter-model disagreement by comparing predictions from two different models to dynamically re-weight loss in training, where a larger disagreement indicates a possible error and corresponds to a lower loss value. Extensive experiments show that DMT achieves state-of-the-art performance in both image classification and semantic segmentation. Our codes are released at https://github.com/voldemortX/DST-CBC .

arXiv commentPublished at Pattern Recognition, see https://www.sciencedirect.com/science/article/abs/pii/S0031320322002588

References in corpus (3)

• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Mixed Precision Training
• Night-time Scene Parsing with a Large Real Dataset

Cited by in corpus (16)

• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• Bootstrapping Semantic Segmentation with Regional Contrast
• Generative Domain Adaptation for Face Anti-Spoofing
• Semi-Supervised Semantic Segmentation Based on Pseudo-Labels: A Survey
• Conservative-Progressive Collaborative Learning for Semi-supervised Semantic Segmentation
• Boosting Night-time Scene Parsing with Learnable Frequency
• Rethinking Domain Generalization: Discriminability and Generalizability
• ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation
• Class-Specific Distribution Alignment for Semi-Supervised Medical Image Classification
• Robust Mutual Learning for Semi-supervised Semantic Segmentation
• Self-Adversarial Disentangling for Specific Domain Adaptation
• PC${}^2$: Pseudo-Classification Based Pseudo-Captioning for Noisy Correspondence Learning in Cross-Modal Retrieval
• SegMatch: A semi-supervised learning method for surgical instrument segmentation
• Multi-Level Label Correction by Distilling Proximate Patterns for Semi-supervised Semantic Segmentation
• SuperCM: Improving Semi-Supervised Learning and Domain Adaptation through differentiable clustering
• About Explicit Variance Minimization: Training Neural Networks for Medical Imaging With Limited Data Annotations