Modular Generative Adversarial Networks

Abstract

Existing methods for multi-domain image-to-image translation (or generation) attempt to directly map an input image (or a random vector) to an image in one of the output domains. However, most existing methods have limited scalability and robustness, since they require building independent models for each pair of domains in question. This leads to two significant shortcomings: (1) the need to train exponential number of pairwise models, and (2) the inability to leverage data from other domains when training a particular pairwise mapping. Inspired by recent work on module networks, this paper proposes ModularGAN for multi-domain image generation and image-to-image translation. ModularGAN consists of several reusable and composable modules that carry on different functions (e.g., encoding, decoding, transformations). These modules can be trained simultaneously, leveraging data from all domains, and then combined to construct specific GAN networks at test time, according to the specific image translation task. This leads to ModularGAN's superior flexibility of generating (or translating to) an image in any desired domain. Experimental results demonstrate that our model not only presents compelling perceptual results but also outperforms state-of-the-art methods on multi-domain facial attribute transfer.

References in corpus (10)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Improved Training of Wasserstein GANs
• Deep Identity-aware Transfer of Facial Attributes
• Learning to Compose Neural Networks for Question Answering
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• Visual Reference Resolution using Attention Memory for Visual Dialog
• Scribbler: Controlling Deep Image Synthesis with Sketch and Color

Cited by in corpus (7)

• ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes
• Fully-Featured Attribute Transfer
• KG-GAN: Knowledge-Guided Generative Adversarial Networks
• FaceShapeGene: A Disentangled Shape Representation for Flexible Face Image Editing
• Multivariate-Information Adversarial Ensemble for Scalable Joint Distribution Matching
• TunaGAN: Interpretable GAN for Smart Editing
• Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks