Joint Face Image Restoration and Frontalization for Recognition

Abstract

In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. To address these challenges, previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. However, most of these methods are stage-wise, which is sub-optimal and deviates from the reality. In this paper, we address all these challenges jointly for unconstrained face recognition. We propose an Multi-Degradation Face Restoration (MDFR) model to restore frontalized high-quality faces from the given low-quality ones under arbitrary facial poses, with three distinct novelties. First, MDFR is a well-designed encoder-decoder architecture which extracts feature representation from an input face image with arbitrary low-quality factors and restores it to a high-quality counterpart. Second, MDFR introduces a pose residual learning strategy along with a 3D-based Pose Normalization Module (PNM), which can perceive the pose gap between the input initial pose and its real-frontal pose to guide the face frontalization. Finally, MDFR can generate frontalized high-quality face images by a single unified network, showing a strong capability of preserving face identity. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks demonstrate the superiority of MDFR over state-of-the-art methods on both face frontalization and face restoration.

arXiv comment14 pages, 9 figures

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising
• Learning Face Representation from Scratch
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• NIMA: Neural Image Assessment
• Examining the Impact of Blur on Recognition by Convolutional Networks
• Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition
• Face Hallucination with Finishing Touches
• Enhance the Motion Cues for Face Anti-Spoofing using CNN-LSTM Architecture
• Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
• When AWGN-based Denoiser Meets Real Noises

Cited by in corpus (4)

• A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces
• CATFace: Cross-Attribute-Guided Transformer with Self-Attention Distillation for Low-Quality Face Recognition
• Deep Learning-based Face Super-Resolution: A Survey
• Inclusive normalization of face images to passport format