Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails

Abstract

In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces (16*16 pixels). Pro-UIGAN iteratively (1) estimates facial geometry priors for low-resolution (LR) faces and (2) acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network super-resolves and inpaints occluded LR faces in a coarse-to-fine manner, thus reducing unwanted blurriness and artifacts significantly. Specifically, we design a novel cross-modal transformer module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutual promotion manner. Extensive experiments demonstrate that our Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks, i.e., face alignment, face parsing, face recognition and expression classification, compared with other state-of-the-art (SotA) methods.

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Pixel Recurrent Neural Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• Deep Cascaded Bi-Network for Face Hallucination
• Attention-Aware Face Hallucination via Deep Reinforcement Learning
• Face Hallucination with Finishing Touches
• Semantically Consistent Image Completion with Fine-grained Details

Cited by in corpus (4)

• CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution
• Deep Learning-based Face Super-Resolution: A Survey
• Automated Deepfake Detection
• Omni-supervised Facial Expression Recognition via Distilled Data