Confidence Regularized Self-Training

Abstract

Recent advances in domain adaptation show that deep self-training presents a powerful means for unsupervised domain adaptation. These methods often involve an iterative process of predicting on target domain and then taking the confident predictions as pseudo-labels for retraining. However, since pseudo-labels can be noisy, self-training can put overconfident label belief on wrong classes, leading to deviated solutions with propagated errors. To address the problem, we propose a confidence regularized self-training (CRST) framework, formulated as regularized self-training. Our method treats pseudo-labels as continuous latent variables jointly optimized via alternating optimization. We propose two types of confidence regularization: label regularization (LR) and model regularization (MR). CRST-LR generates soft pseudo-labels while CRST-MR encourages the smoothness on network output. Extensive experiments on image classification and semantic segmentation show that CRSTs outperform their non-regularized counterpart with state-of-the-art performance. The code and models of this work are available at https://github.com/yzou2/CRST.

Accepted to ICCV 2019 (Oral)

References in corpus (16)

• Distilling the Knowledge in a Neural Network
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• On Calibration of Modern Neural Networks
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Conditional Adversarial Domain Adaptation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Training Convolutional Networks with Noisy Labels
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Label Refinery: Improving ImageNet Classification through Label Progression
• Unsupervised Domain Adaptation via Calibrating Uncertainties

Cited by in corpus (7)

• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
• Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models
• Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation
• VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
• PØDA: Prompt-driven Zero-shot Domain Adaptation
• Unsupervised Domain Expansion for Visual Categorization