Cross-Domain Latent Modulation for Variational Transfer Learning

Abstract

We propose a cross-domain latent modulation mechanism within a variational autoencoders (VAE) framework to enable improved transfer learning. Our key idea is to procure deep representations from one data domain and use it as perturbation to the reparameterization of the latent variable in another domain. Specifically, deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. Second, the learned deep representations are cross-modulated to the latent encoding of the alternate domain. The consistency between the reconstruction from the modulated latent encoding and the generation using deep representation samples is then enforced in order to produce inter-class alignment in the latent space. We apply the proposed model to a number of transfer learning tasks including unsupervised domain adaptation and image-toimage translation. Experimental results show that our model gives competitive performance.

10 pages, 7 figures, to appear in IEEE WACV'21

References in corpus (14)

• Auto-Encoding Variational Bayes
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• How transferable are features in deep neural networks?
• Learning Transferable Features with Deep Adaptation Networks
• Conditional Adversarial Domain Adaptation
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Domain Separation Networks
• Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
• A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation
• Cross-Domain Transferability of Adversarial Perturbations
• Latent Normalizing Flows for Discrete Sequences
• Fast Gradient-Based Inference with Continuous Latent Variable Models in Auxiliary Form
• Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings