Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings
arXiv:2002.06661
Abstract
Learned joint representations of images and text form the backbone of several important cross-domain tasks such as image captioning. Prior work mostly maps both domains into a common latent representation in a purely supervised fashion. This is rather restrictive, however, as the two domains follow distinct generative processes. Therefore, we propose a novel semi-supervised framework, which models shared information between domains and domain-specific information separately. The information shared between the domains is aligned with an invertible neural network. Our model integrates normalizing flow-based priors for the domain-specific information, which allows us to learn diverse many-to-many mappings between the two domains. We demonstrate the effectiveness of our model on diverse tasks, including image captioning and text-to-image synthesis.
Published as a conference paper at ICLR 2020
References in corpus (14)
- Auto-Encoding Variational Bayes
- Neural Discrete Representation Learning
- Self-Attention Generative Adversarial Networks
- Improved Techniques for Training GANs
- Variational Lossy Autoencoder
- Analyzing Inverse Problems with Invertible Neural Networks
- TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network
- Towards Diverse and Natural Image Descriptions via a Conditional GAN
- MirrorGAN: Learning Text-to-image Generation by Redescription
- Latent Normalizing Flows for Discrete Sequences
- Generating Diverse and Accurate Visual Captions by Comparative Adversarial Learning
- Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
- Semantics Disentangling for Text-to-Image Generation
- M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention