Semantically Decomposing the Latent Spaces of Generative Adversarial Networks

Abstract

We propose a new algorithm for training generative adversarial networks that jointly learns latent codes for both identities (e.g. individual humans) and observations (e.g. specific photographs). By fixing the identity portion of the latent codes, we can generate diverse images of the same subject, and by fixing the observation portion, we can traverse the manifold of subjects while maintaining contingent aspects such as lighting and pose. Our algorithm features a pairwise training scheme in which each sample from the generator consists of two images with a common identity code. Corresponding samples from the real dataset consist of two distinct photographs of the same subject. In order to fool the discriminator, the generator must produce pairs that are photorealistic, distinct, and appear to depict the same individual. We augment both the DCGAN and BEGAN approaches with Siamese discriminators to facilitate pairwise training. Experiments with human judges and an off-the-shelf face verification system demonstrate our algorithm's ability to generate convincing, identity-matched photographs.

Published as a conference paper at ICLR 2018

References in corpus (9)

• Conditional Generative Adversarial Nets
• Learning Face Representation from Scratch
• Energy-based Generative Adversarial Network
• Deep Convolutional Inverse Graphics Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Unsupervised Image-to-Image Translation Networks
• Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
• Multi-View Image Generation from a Single-View
• Semi-Latent GAN: Learning to generate and modify facial images from attributes

Cited by in corpus (11)

• How Generative Adversarial Networks and Their Variants Work: An Overview
• Generative Adversarial Networks for Spatio-temporal Data: A Survey
• Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
• Training Invertible Neural Networks as Autoencoders
• Recent Progress of Face Image Synthesis
• Improved Techniques for GAN based Facial Inpainting
• Shadow Transfer: Single Image Relighting For Urban Road Scenes
• Taking Control of Intra-class Variation in Conditional GANs Under Weak Supervision
• Semantic Interpolation in Implicit Models
• Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation
• Training face verification models from generated face identity data