Neural Face Editing with Intrinsic Image Disentangling

Abstract

Traditional face editing methods often require a number of sophisticated and task specific algorithms to be applied one after the other --- a process that is tedious, fragile, and computationally intensive. In this paper, we propose an end-to-end generative adversarial network that infers a face-specific disentangled representation of intrinsic face properties, including shape (i.e. normals), albedo, and lighting, and an alpha matte. We show that this network can be trained on "in-the-wild" images by incorporating an in-network physically-based image formation module and appropriate loss functions. Our disentangling latent representation allows for semantically relevant edits, where one aspect of facial appearance can be manipulated while keeping orthogonal properties fixed, and we demonstrate its use for a number of facial editing applications.

arXiv commentCVPR 2017 oral

References in corpus (5)

• Energy-based Generative Adversarial Network
• Deep Convolutional Inverse Graphics Network
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Do We Really Need to Collect Millions of Faces for Effective Face Recognition?
• Deep Manifold Traversal: Changing Labels with Convolutional Features

Cited by in corpus (21)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• ShaRF: Shape-conditioned Radiance Fields from a Single View
• Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
• SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
• Facelet-Bank for Fast Portrait Manipulation
• Disentangling Factors of Variation by Mixing Them
• SharinGAN: Combining Synthetic and Real Data for Unsupervised Geometry Estimation
• Differential Generative Adversarial Networks: Synthesizing Non-linear Facial Variations with Limited Number of Training Data
• Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN
• InverseRenderNet: Learning single image inverse rendering
• Learning to Reconstruct Texture-less Deformable Surfaces from a Single View
• Instance-level Facial Attributes Transfer with Geometry-Aware Flow
• GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
• Towards Interpretable Face Recognition
• Dense Pose Transfer
• Variational Capsules for Image Analysis and Synthesis
• Texture Deformation Based Generative Adversarial Networks for Face Editing
• Disentangling Content and Style via Unsupervised Geometry Distillation
• Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation
• CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering
• Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction