Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization

Abstract

Face frontalization refers to the process of synthesizing the frontal view of a face from a given profile. Due to self-occlusion and appearance distortion in the wild, it is extremely challenging to recover faithful results and preserve texture details in a high-resolution. This paper proposes a High Fidelity Pose Invariant Model (HF-PIM) to produce photographic and identity-preserving results. HF-PIM frontalizes the profiles through a novel texture warping procedure and leverages a dense correspondence field to bind the 2D and 3D surface spaces. We decompose the prerequisite of warping into dense correspondence field estimation and facial texture map recovering, which are both well addressed by deep networks. Different from those reconstruction methods relying on 3D data, we also propose Adversarial Residual Dictionary Learning (ARDL) to supervise facial texture map recovering with only monocular images. Exhaustive experiments on both controlled and uncontrolled environments demonstrate that the proposed method not only boosts the performance of pose-invariant face recognition but also dramatically improves high-resolution frontalization appearances.

To appear in NIPS 2018

Cited by in corpus (13)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• OSTeC: One-Shot Texture Completion
• Towards Fast, Accurate and Stable 3D Dense Face Alignment
• Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images
• Textured Neural Avatars
• Heterogeneous Face Frontalization via Domain Agnostic Learning
• M2FPA: A Multi-Yaw Multi-Pitch High-Quality Database and Benchmark for Facial Pose Analysis
• Biphasic Learning of GANs for High-Resolution Image-to-Image Translation
• Dual-Attention GAN for Large-Pose Face Frontalization
• Learning Flow-based Feature Warping for Face Frontalization with Illumination Inconsistent Supervision
• LGLG-WPCA: An Effective Texture-based Method for Face Recognition
• Free-Form Image Inpainting via Contrastive Attention Network
• Cosmetic-Aware Makeup Cleanser