3D Face Reconstruction with Geometry Details from a Single Image

Abstract

3D face reconstruction from a single image is a classical and challenging problem, with wide applications in many areas. Inspired by recent works in face animation from RGB-D or monocular video inputs, we develop a novel method for reconstructing 3D faces from unconstrained 2D images, using a coarse-to-fine optimization strategy. First, a smooth coarse 3D face is generated from an example-based bilinear face model, by aligning the projection of 3D face landmarks with 2D landmarks detected from the input image. Afterwards, using local corrective deformation fields, the coarse 3D face is refined using photometric consistency constraints, resulting in a medium face shape. Finally, a shape-from-shading method is applied on the medium face to recover fine geometric details. Our method outperforms state-of-the-art approaches in terms of accuracy and detail recovery, which is demonstrated in extensive experiments using real world models and publicly available datasets.

Accepted by IEEE Transactions on Image Processing, 2018

Cited by in corpus (23)

• GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction
• Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose
• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction
• Prior-Guided Multi-View 3D Head Reconstruction
• 3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos
• Photo-Realistic Facial Details Synthesis from Single Image
• Adaptive Joint Optimization for 3D Reconstruction with Differentiable Rendering
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape
• Deep Deformable 3D Caricatures with Learned Shape Control
• InverseFaceNet: Deep Monocular Inverse Face Rendering
• Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions
• Face De-occlusion using 3D Morphable Model and Generative Adversarial Network
• Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture
• Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model
• CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images
• 3D Magic Mirror: Automatic Video to 3D Caricature Translation
• Personalized Face Modeling for Improved Face Reconstruction and Motion Retargeting
• 3D Face Reconstruction Using Color Photometric Stereo with Uncalibrated Near Point Lights
• End-to-end 3D shape inverse rendering of different classes of objects from a single input image
• Lightweight Photometric Stereo for Facial Details Recovery
• Adaptive 3D Face Reconstruction from a Single Image
• High Accuracy Face Geometry Capture using a Smartphone Video
• A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction