Transfer Learning with Dynamic Adversarial Adaptation Network

Abstract

The recent advances in deep transfer learning reveal that adversarial learning can be embedded into deep networks to learn more transferable features to reduce the distribution discrepancy between two domains. Existing adversarial domain adaptation methods either learn a single domain discriminator to align the global source and target distributions or pay attention to align subdomains based on multiple discriminators. However, in real applications, the marginal (global) and conditional (local) distributions between domains are often contributing differently to the adaptation. There is currently no method to dynamically and quantitatively evaluate the relative importance of these two distributions for adversarial learning. In this paper, we propose a novel Dynamic Adversarial Adaptation Network (DAAN) to dynamically learn domain-invariant representations while quantitatively evaluate the relative importance of global and local domain distributions. To the best of our knowledge, DAAN is the first attempt to perform dynamic adversarial distribution adaptation for deep adversarial learning. DAAN is extremely easy to implement and train in real applications. We theoretically analyze the effectiveness of DAAN, and it can also be explained in an attention strategy. Extensive experiments demonstrate that DAAN achieves better classification accuracy compared to state-of-the-art deep and adversarial methods. Results also imply the necessity and effectiveness of the dynamic distribution adaptation in adversarial transfer learning.

ICDM 2019 long paper (9.08% acceptance rate); 9 pages; code available at http://transferlearning.xyz

References in corpus (10)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• How transferable are features in deep neural networks?
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Multi-Representation Adaptation Network for Cross-domain Image Classification
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Co-regularized Alignment for Unsupervised Domain Adaptation
• Accelerating Deep Unsupervised Domain Adaptation with Transfer Channel Pruning

Cited by in corpus (1)

• Unsupervised Domain Adaptation with Progressive Domain Augmentation