Taking Control of Intra-class Variation in Conditional GANs Under Weak Supervision

Abstract

Generative Adversarial Networks (GANs) are able to learn mappings between simple, relatively low-dimensional, random distributions and points on the manifold of realistic images in image-space. The semantics of this mapping, however, are typically entangled such that meaningful image properties cannot be controlled independently of one another. Conditional GANs (cGANs) provide a potential solution to this problem, allowing specific semantics to be enforced during training. This solution, however, depends on the availability of precise labels, which are sometimes difficult or near impossible to obtain, e.g. labels representing lighting conditions or describing the background. In this paper we introduce a new formulation of the cGAN that is able to learn disentangled, multivariate models of semantically meaningful variation and which has the advantage of requiring only the weak supervision of binary attribute labels. For example, given only labels of ambient / non-ambient lighting, our method is able to learn multivariate lighting models disentangled from other factors such as the identity and pose. We coin the method intra-class variation isolation (IVI) and the resulting network the IVI-GAN. We evaluate IVI-GAN on the CelebA dataset and on synthetic 3D morphable model data, learning to disentangle attributes such as lighting, pose, expression, and even the background.

References in corpus (12)

• Conditional Generative Adversarial Nets
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• A Learned Representation For Artistic Style
• Invertible Conditional GANs for image editing
• cGANs with Projection Discriminator
• Neural Photo Editing with Introspective Adversarial Networks
• Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
• High Resolution Face Completion with Multiple Controllable Attributes via Fully End-to-End Progressive Generative Adversarial Networks
• Differential Generative Adversarial Networks: Synthesizing Non-linear Facial Variations with Limited Number of Training Data

Cited by in corpus (2)

• An Assessment of GANs for Identity-related Applications
• On the use of automatically generated synthetic image datasets for benchmarking face recognition