Characterizing and Avoiding Negative Transfer
arXiv:1811.09751
Abstract
When labeled data is scarce for a specific target task, transfer learning often offers an effective solution by utilizing data from a related source task. However, when transferring knowledge from a less related source, it may inversely hurt the target performance, a phenomenon known as negative transfer. Despite its pervasiveness, negative transfer is usually described in an informal manner, lacking rigorous definition, careful analysis, or systematic treatment. This paper proposes a formal definition of negative transfer and analyzes three important aspects thereof. Stemming from this analysis, a novel technique is proposed to circumvent negative transfer by filtering out unrelated source data. Based on adversarial networks, the technique is highly generic and can be applied to a wide range of transfer learning algorithms. The proposed approach is evaluated on six state-of-the-art deep transfer methods via experiments on four benchmark datasets with varying levels of difficulty. Empirically, the proposed method consistently improves the performance of all baseline methods and largely avoids negative transfer, even when the source data is degenerate.
arXiv commentPublished at CVPR 2019
References in corpus (9)
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- How transferable are features in deep neural networks?
- Learning Transferable Features with Deep Adaptation Networks
- VisDA: The Visual Domain Adaptation Challenge
- Deep CORAL: Correlation Alignment for Deep Domain Adaptation
- Why do deep convolutional networks generalize so poorly to small image transformations?
- Analysis of Kernel Mean Matching under Covariate Shift
- Generative Adversarial Nets from a Density Ratio Estimation Perspective
- Partial Adversarial Domain Adaptation