SiGAN: Siamese Generative Adversarial Network for Identity-Preserving Face Hallucination

Abstract

Despite generative adversarial networks (GANs) can hallucinate photo-realistic high-resolution (HR) faces from low-resolution (LR) faces, they cannot guarantee preserving the identities of hallucinated HR faces, making the HR faces poorly recognizable. To address this problem, we propose a Siamese GAN (SiGAN) to reconstruct HR faces that visually resemble their corresponding identities. On top of a Siamese network, the proposed SiGAN consists of a pair of two identical generators and one discriminator. We incorporate reconstruction error and identity label information in the loss function of SiGAN in a pairwise manner. By iteratively optimizing the loss functions of the generator pair and discriminator of SiGAN, we cannot only achieve photo-realistic face reconstruction, but also ensures the reconstructed information is useful for identity recognition. Experimental results demonstrate that SiGAN significantly outperforms existing face hallucination GANs in objective face verification performance, while achieving photo-realistic reconstruction. Moreover, for input LR faces from unknown identities who are not included in training, SiGAN can still do a good job.

13 pages

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Learning Face Representation from Scratch
• Robust Face Recognition via Multimodal Deep Face Representation
• Surveillance Face Recognition Challenge
• Deep Cascaded Bi-Network for Face Hallucination
• Attention-Aware Face Hallucination via Deep Reinforcement Learning
• Deep Joint Face Hallucination and Recognition
• Learning to Hallucinate Face Images via Component Generation and Enhancement
• Global-Local Face Upsampling Network

Cited by in corpus (13)

• Generating Visually Aligned Sound from Videos
• E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
• Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition
• Edge and Identity Preserving Network for Face Super-Resolution
• PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier
• F3A-GAN: Facial Flow for Face Animation with Generative Adversarial Networks
• JDSR-GAN: Constructing An Efficient Joint Learning Network for Masked Face Super-Resolution
• Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space with Softplus Loss
• Channel adversarial training for speaker verification and diarization
• Octuplet Loss: Make Face Recognition Robust to Image Resolution
• Relating by Contrasting: A Data-efficient Framework for Multimodal Generative Models
• Deep Learning-based Face Super-Resolution: A Survey
• Neural Alignment for Face De-pixelization