Photo-Realistic Facial Details Synthesis from Single Image

Abstract

We present a single-image 3D face synthesis technique that can handle challenging facial expressions while recovering fine geometric details. Our technique employs expression analysis for proxy face geometry generation and combines supervised and unsupervised learning for facial detail synthesis. On proxy generation, we conduct emotion prediction to determine a new expression-informed proxy. On detail synthesis, we present a Deep Facial Detail Net (DFDN) based on Conditional Generative Adversarial Net (CGAN) that employs both geometry and appearance loss functions. For geometry, we capture 366 high-quality 3D scans from 122 different subjects under 3 facial expressions. For appearance, we use additional 20K in-the-wild face images and apply image-based rendering to accommodate lighting variations. Comprehensive experiments demonstrate that our framework can produce high-quality 3D faces with realistic details under challenging facial expressions.

References in corpus (5)

• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• 3D Face Reconstruction with Geometry Details from a Single Image
• Real-time Convolutional Neural Networks for Emotion and Gender Classification
• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
• Unsupervised Training for 3D Morphable Model Regression

Cited by in corpus (7)

• FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction
• Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networks
• Vision and Language: from Visual Perception to Content Creation
• Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency
• Cross-modal Deep Face Normals with Deactivable Skip Connections
• FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction
• Lightweight Photometric Stereo for Facial Details Recovery