Towards Large-Pose Face Frontalization in the Wild

Abstract

Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments. Learning pose-invariant features is one solution, but needs expensively labeled large-scale data and carefully designed feature learning algorithms. In this work, we focus on frontalizing faces in the wild under various head poses, including extreme profile views. We propose a novel deep 3D Morphable Model (3DMM) conditioned Face Frontalization Generative Adversarial Network (GAN), termed as FF-GAN, to generate neutral head pose face images. Our framework differs from both traditional GANs and 3DMM based modeling. Incorporating 3DMM into the GAN structure provides shape and appearance priors for fast convergence with less training data, while also supporting end-to-end training. The 3DMM-conditioned GAN employs not only the discriminator and generator loss but also a new masked symmetry loss to retain visual quality under occlusions, besides an identity loss to recover high frequency information. Experiments on face recognition, landmark localization and 3D reconstruction consistently show the advantage of our frontalization method on faces in the wild datasets.

To appear at ICCV2017. Details refer to http://cvlab.cse.msu.edu/project-face-frontalization.html

References in corpus (6)

• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
• Pose-Invariant 3D Face Alignment
• Reconstruction-Based Disentanglement for Pose-invariant Face Recognition

Cited by in corpus (19)

• Deep Face Recognition: A Survey
• Geometry-Contrastive GAN for Facial Expression Transfer
• Towards Open-Set Identity Preserving Face Synthesis
• A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion
• Feature Transfer Learning for Deep Face Recognition with Under-Represented Data
• Nonlinear 3D Face Morphable Model
• Unsupervised Depth Estimation, 3D Face Rotation and Replacement
• Identity Preserving Face Completion for Large Ocular Region Occlusion
• Triple consistency loss for pairing distributions in GAN-based face synthesis
• Recent Progress of Face Image Synthesis
• Towards Interpretable Face Recognition
• Landmark Weighting for 3DMM Shape Fitting
• GridFace: Face Rectification via Learning Local Homography Transformations
• Pipeline Generative Adversarial Networks for Facial Images Generation with Multiple Attributes
• DotFAN: A Domain-transferred Face Augmentation Network for Pose and Illumination Invariant Face Recognition
• When 3D-Aided 2D Face Recognition Meets Deep Learning: An extended UR2D for Pose-Invariant Face Recognition
• Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
• Inner Space Preserving Generative Pose Machine
• On Improving the Generalization of Face Recognition in the Presence of Occlusions