GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling
arXiv:2003.06788
Abstract
Unsupervised image-to-image translation (UNIT) aims at learning a mapping between several visual domains by using unpaired training images. Recent studies have shown remarkable success for multiple domains but they suffer from two main limitations: they are either built from several two-domain mappings that are required to be learned independently, or they generate low-diversity results, a problem known as mode collapse. To overcome these limitations, we propose a method named GMM-UNIT, which is based on a content-attribute disentangled representation where the attribute space is fitted with a GMM. Each GMM component represents a domain, and this simple assumption has two prominent advantages. First, it can be easily extended to most multi-domain and multi-modal image-to-image translation tasks. Second, the continuous domain encoding allows for interpolation between domains and for extrapolation to unseen domains and translations. Additionally, we show how GMM-UNIT can be constrained down to different methods in the literature, meaning that GMM-UNIT is a unifying framework for unsupervised image-to-image translation.
arXiv comment27 pages, 17 figures
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Toward Multimodal Image-to-Image Translation
- Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
- Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
- DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
Cited by in corpus (4)
- Image-to-Image Translation: Methods and Applications
- Domain-Scalable Unpaired Image Translation via Latent Space Anchoring
- SDA-GAN: Unsupervised Image Translation Using Spectral Domain Attention-Guided Generative Adversarial Network
- Variational Bayesian Framework for Advanced Image Generation with Domain-Related Variables