Interpreting the Latent Space of GANs for Semantic Face Editing

Abstract

Despite the recent advance of Generative Adversarial Networks (GANs) in high-fidelity image synthesis, there lacks enough understanding of how GANs are able to map a latent code sampled from a random distribution to a photo-realistic image. Previous work assumes the latent space learned by GANs follows a distributed representation but observes the vector arithmetic phenomenon. In this work, we propose a novel framework, called InterFaceGAN, for semantic face editing by interpreting the latent semantics learned by GANs. In this framework, we conduct a detailed study on how different semantics are encoded in the latent space of GANs for face synthesis. We find that the latent code of well-trained generative models actually learns a disentangled representation after linear transformations. We explore the disentanglement between various semantics and manage to decouple some entangled semantics with subspace projection, leading to more precise control of facial attributes. Besides manipulating gender, age, expression, and the presence of eyeglasses, we can even vary the face pose as well as fix the artifacts accidentally generated by GAN models. The proposed method is further applied to achieve real image manipulation when combined with GAN inversion methods or some encoder-involved models. Extensive results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable facial attribute representation.

CVPR2020 camera-ready

References in corpus (12)

• Conditional Generative Adversarial Nets
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Invertible Conditional GANs for image editing
• Fader Networks: Manipulating Images by Sliding Attributes
• Optimizing the Latent Space of Generative Networks
• Metrics for Deep Generative Models
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• FaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face Synthesis
• Latent Space Non-Linear Statistics

Cited by in corpus (23)

• GANSpace: Discovering Interpretable GAN Controls
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• Collaborative Learning for Faster StyleGAN Embedding
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• On the "steerability" of generative adversarial networks
• FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
• Using latent space regression to analyze and leverage compositionality in GANs
• Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space
• Feature Unlearning for Pre-trained GANs and VAEs
• Editing in Style: Uncovering the Local Semantics of GANs
• MammoGANesis: Controlled Generation of High-Resolution Mammograms for Radiology Education
• Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
• CMTS: Conditional Multiple Trajectory Synthesizer for Generating Safety-critical Driving Scenarios
• Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation
• DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling
• Learning Disentangled Representations with Latent Variation Predictability
• GAN-Based Facial Attractiveness Enhancement
• Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation
• Image Translation by Latent Union of Subspaces for Cross-Domain Plaque Detection
• Neuro-Symbolic Generative Art: A Preliminary Study
• Inverting Generative Adversarial Renderer for Face Reconstruction
• AE-StyleGAN: Improved Training of Style-Based Auto-Encoders
• DeFLOCNet: Deep Image Editing via Flexible Low-level Controls