Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

Abstract

We propose a straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment. To achieve this, we design a 2D representation called UV position map which records the 3D shape of a complete face in UV space, then train a simple Convolutional Neural Network to regress it from a single 2D image. We also integrate a weight mask into the loss function during training to improve the performance of the network. Our method does not rely on any prior face model, and can reconstruct full facial geometry along with semantic meaning. Meanwhile, our network is very light-weighted and spends only 9.8ms to process an image, which is extremely faster than previous works. Experiments on multiple challenging datasets show that our method surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.

18 pages, 10 figures

References in corpus (12)

• 3D Morphable Models as Spatial Transformer Networks
• A Point Set Generation Network for 3D Object Reconstruction from a Single Image
• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
• Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks
• Pose-Invariant 3D Face Alignment
• Real-Time Facial Segmentation and Performance Capture from RGB Input
• Learning Dense Facial Correspondences in Unconstrained Images
• Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network
• Multi-Attribute Robust Component Analysis for Facial UV Maps
• UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition
• Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks
• Side Information for Face Completion: a Robust PCA Approach

Cited by in corpus (18)

• Learning Meta Model for Zero- and Few-shot Face Anti-spoofing
• Exploiting temporal and depth information for multi-frame face anti-spoofing
• Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark
• Aurora Guard: Real-Time Face Anti-Spoofing via Light Reflection
• Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing
• A Pixel-Based Framework for Data-Driven Clothing
• FaceX-Zoo: A PyTorch Toolbox for Face Recognition
• Aurora Guard: Reliable Face Anti-Spoofing via Mobile Lighting System
• Masked Face Recognition Challenge: The InsightFace Track Report
• Static and Dynamic Fusion for Multi-modal Cross-ethnicity Face Anti-spoofing
• Deep Learning-based Face Super-Resolution: A Survey
• DBLFace: Domain-Based Labels for NIR-VIS Heterogeneous Face Recognition
• End to End Face Reconstruction via Differentiable PnP
• Synthetic Expressions are Better Than Real for Learning to Detect Facial Actions
• Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction
• Think about boundary: Fusing multi-level boundary information for landmark heatmap regression
• Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion
• Learning Robust 3D Face Reconstruction and Discriminative Identity Representation