Transfer Learning with Label Noise

Abstract

Transfer learning aims to improve learning in target domain by borrowing knowledge from a related but different source domain. To reduce the distribution shift between source and target domains, recent methods have focused on exploring invariant representations that have similar distributions across domains. However, when learning this invariant knowledge, existing methods assume that the labels in source domain are uncontaminated, while in reality, we often have access to source data with noisy labels. In this paper, we first show how label noise adversely affect the learning of invariant representations and the correcting of label shift in various transfer learning scenarios. To reduce the adverse effects, we propose a novel Denoising Conditional Invariant Component (DCIC) framework, which provably ensures (1) extracting invariant representations given examples with noisy labels in source domain and unlabeled examples in target domain; (2) estimating the label distribution in target domain with no bias. Experimental results on both synthetic and real-world data verify the effectiveness of the proposed method.

References in corpus (5)

• Learning Transferable Features with Deep Adaptation Networks
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• VisDA: The Visual Domain Adaptation Challenge
• Training Convolutional Networks with Noisy Labels
• On Causal and Anticausal Learning

Cited by in corpus (10)

• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Instance-Dependent PU Learning by Bayesian Optimal Relabeling
• Penalty Method for Inversion-Free Deep Bilevel Optimization
• DistillHash: Unsupervised Deep Hashing by Distilling Data Pairs
• Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation
• Learning with Biased Complementary Labels
• Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
• SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels
• Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy
• Millionaire: A Hint-guided Approach for Crowdsourcing