Edge and Identity Preserving Network for Face Super-Resolution

Abstract

Face super-resolution (SR) has become an indispensable function in security solutions such as video surveillance and identification system, but the distortion in facial components is a great challenge in it. Most state-of-the-art methods have utilized facial priors with deep neural networks. These methods require extra labels, longer training time, and larger computation memory. In this paper, we propose a novel Edge and Identity Preserving Network for Face SR Network, named as EIPNet, to minimize the distortion by utilizing a lightweight edge block and identity information. We present an edge block to extract perceptual edge information, and concatenate it to the original feature maps in multiple scales. This structure progressively provides edge information in reconstruction to aggregate local and global structural information. Moreover, we define an identity loss function to preserve identification of SR images. The identity loss function compares feature distributions between SR images and their ground truth to recover identities in SR images. In addition, we provide a luminance-chrominance error (LCE) to separately infer brightness and color information in SR images. The LCE method not only reduces the dependency of color information by dividing brightness and color components but also enables our network to reflect differences between SR images and their ground truth in two color spaces of RGB and YUV. The proposed method facilitates the proposed SR network to elaborately restore facial components and generate high quality 8x scaled SR images with a lightweight network structure. Furthermore, our network is able to reconstruct an 128x128 SR image with 215 fps on a GTX 1080Ti GPU. Extensive experiments demonstrate that our network qualitatively and quantitatively outperforms state-of-the-art methods on two challenging datasets: CelebA and VGGFace2.

arXiv commentNeurocomputing'2021

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• TensorFlow: A system for large-scale machine learning
• The Perception-Distortion Tradeoff
• EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
• Progressive Face Super-Resolution via Attention to Facial Landmark
• SiGAN: Siamese Generative Adversarial Network for Identity-Preserving Face Hallucination
• SREdgeNet: Edge Enhanced Single Image Super Resolution using Dense Edge Detection Network and Feature Merge Network
• EPSR: Edge Profile Super resolution
• Deep CNN Denoiser and Multi-layer Neighbor Component Embedding for Face Hallucination

Cited by in corpus (5)

• Octuplet Loss: Make Face Recognition Robust to Image Resolution
• Color Models in Image Processing: A Review and Experimental Comparison
• Deep Learning-based Face Super-Resolution: A Survey
• Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors
• Improving Generative Adversarial Network Generalization for Facial Expression Synthesis