Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

Abstract

3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination. In general these methods require complex and inefficient pipelines for model building and fitting. In this work, we propose to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans. Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model. We achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions. Testing code will be made available online, along with pre-trained models http://aaronsplace.co.uk/papers/jackson2017recon

10 pages, ICCV 2017

References in corpus (1)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Cited by in corpus (23)

• Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
• 3D Morphable Models as Spatial Transformer Networks
• BodyNet: Volumetric Inference of 3D Human Body Shapes
• Generating 3D faces using Convolutional Mesh Autoencoders
• Photo-Realistic Facial Details Synthesis from Single Image
• Unsupervised Depth Estimation, 3D Face Rotation and Replacement
• Unsupervised Training for 3D Morphable Model Regression
• CaricatureShop: Personalized and Photorealistic Caricature Sketching
• InverseFaceNet: Deep Monocular Inverse Face Rendering
• Deep Learning-based Face Super-Resolution: A Survey
• 3D Face Anti-spoofing with Factorized Bilinear Coding
• CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images
• Extreme 3D Face Reconstruction: Seeing Through Occlusions
• 3D Face Reconstruction from Light Field Images: A Model-free Approach
• HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks
• Sparse Photometric 3D Face Reconstruction Guided by Morphable Models
• End-to-end 3D shape inverse rendering of different classes of objects from a single input image
• Deep Structure for end-to-end inverse rendering
• High-Quality Face Capture Using Anatomical Muscles
• Learning Robust 3D Face Reconstruction and Discriminative Identity Representation
• MobileFace: 3D Face Reconstruction with Efficient CNN Regression
• A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction
• Fast Face Image Synthesis with Minimal Training