Cross-Domain Latent Modulation for Variational Transfer Learning

Abstract

We propose a cross-domain latent modulation mechanism within a variational autoencoders (VAE) framework to enable improved transfer learning. Our key idea is to procure deep representations from one data domain and use it as perturbation to the reparameterization of the latent variable in another domain. Specifically, deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. Second, the learned deep representations are cross-modulated to the latent encoding of the alternate domain. The consistency between the reconstruction from the modulated latent encoding and the generation using deep representation samples is then enforced in order to produce inter-class alignment in the latent space. We apply the proposed model to a number of transfer learning tasks including unsupervised domain adaptation and image-toimage translation. Experimental results show that our model gives competitive performance.

10 pages, 7 figures, to appear in IEEE WACV'21

References in corpus (6)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• How transferable are features in deep neural networks?
• Learning Transferable Features with Deep Adaptation Networks
• Domain Separation Networks
• Latent Normalizing Flows for Discrete Sequences
• Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings