InverseFaceNet: Deep Monocular Inverse Face Rendering

Abstract

We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible in real time. Most previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. We further propose a self-supervised bootstrapping process in the network training loop, which iteratively updates the synthetic training corpus to better reflect the distribution of real-world imagery. We demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.

CVPR 2018 (poster) 10 pages (+5 pages)

References in corpus (10)

• ADADELTA: An Adaptive Learning Rate Method
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Deep Convolutional Inverse Graphics Network
• 3D Face Reconstruction with Geometry Details from a Single Image
• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
• A Recurrent Encoder-Decoder Network for Sequential Face Alignment
• Robust LSTM-Autoencoders for Face De-Occlusion in the Wild
• A Visual Representation for Editing Face Images
• Face Alignment In-the-Wild: A Survey
• 3D Face Morphable Models "In-the-Wild"

Cited by in corpus (9)

• 3D Morphable Models as Spatial Transformer Networks
• Recent Progress of Face Image Synthesis
• Multi-Attribute Robust Component Analysis for Facial UV Maps
• A Hybrid Model for Identity Obfuscation by Face Replacement
• Learning Perspective Undistortion of Portraits
• Informed MCMC with Bayesian Neural Networks for Facial Image Analysis
• UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition
• End-to-end 3D shape inverse rendering of different classes of objects from a single input image
• A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction