Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis

Abstract

Photorealistic frontal view synthesis from a single face image has a wide range of applications in the field of face recognition. Although data-driven deep learning methods have been proposed to address this problem by seeking solutions from ample face data, this problem is still challenging because it is intrinsically ill-posed. This paper proposes a Two-Pathway Generative Adversarial Network (TP-GAN) for photorealistic frontal view synthesis by simultaneously perceiving global structures and local details. Four landmark located patch networks are proposed to attend to local textures in addition to the commonly used global encoder-decoder network. Except for the novel architecture, we make this ill-posed problem well constrained by introducing a combination of adversarial loss, symmetry loss and identity preserving loss. The combined loss function leverages both frontal face distribution and pre-trained discriminative deep face models to guide an identity preserving inference of frontal views from profiles. Different from previous deep learning methods that mainly rely on intermediate features for recognition, our method directly leverages the synthesized identity preserving image for downstream tasks like face recognition and attribution estimation. Experimental results demonstrate that our method not only presents compelling perceptual results but also outperforms state-of-the-art results on large pose face recognition.

accepted at ICCV 2017, main paper & supplementary material, 11 pages

References in corpus (6)

• Conditional Generative Adversarial Nets
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro
• MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
• Deep Joint Face Hallucination and Recognition
• Learning Residual Images for Face Attribute Manipulation

Cited by in corpus (44)

• Deep Facial Expression Recognition: A Survey
• How Generative Adversarial Networks and Their Variants Work: An Overview
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• An Introduction to Image Synthesis with Generative Adversarial Nets
• On Learning 3D Face Morphable Model from In-the-wild Images
• CR-GAN: Learning Complete Representations for Multi-view Generation
• Attributes Guided Feature Learning for Vehicle Re-identification
• Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
• Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition
• Feature Transfer Learning for Deep Face Recognition with Under-Represented Data
• Unsupervised Depth Estimation, 3D Face Rotation and Replacement
• Load Balanced GANs for Multi-view Face Image Synthesis
• Unsupervised Person Image Synthesis in Arbitrary Poses
• Exploring Disentangled Feature Representation Beyond Face Identification
• Triple consistency loss for pairing distributions in GAN-based face synthesis
• Recent Progress of Face Image Synthesis
• Adversarial Discriminative Heterogeneous Face Recognition
• Disentangled Variational Representation for Heterogeneous Face Recognition
• Dual Variational Generation for Low-Shot Heterogeneous Face Recognition
• What Is It Like Down There? Generating Dense Ground-Level Views and Image Features From Overhead Imagery Using Conditional Generative Adversarial Networks
• Synthesizing facial photometries and corresponding geometries using generative adversarial networks
• UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition
• View Extrapolation of Human Body from a Single Image
• GridFace: Face Rectification via Learning Local Homography Transformations
• Biphasic Learning of GANs for High-Resolution Image-to-Image Translation
• Geometry-Aware Face Completion and Editing
• Disentanglement for Discriminative Visual Recognition
• Ebola Optimization Search Algorithm (EOSA): A new metaheuristic algorithm based on the propagation model of Ebola virus disease
• Adversarial Occlusion-aware Face Detection
• FD-GAN: Face-demorphing generative adversarial network for restoring accomplice's facial image
• FaceShapeGene: A Disentangled Shape Representation for Flexible Face Image Editing
• GAN-Based Facial Attractiveness Enhancement
• Learning Symmetry Consistent Deep CNNs for Face Completion
• DotFAN: A Domain-transferred Face Augmentation Network for Pose and Illumination Invariant Face Recognition
• Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
• Side Information for Face Completion: a Robust PCA Approach
• Specular-to-Diffuse Translation for Multi-View Reconstruction
• Cross-View Image Synthesis with Deformable Convolution and Attention Mechanism
• Adversarial View-Consistent Learning for Monocular Depth Estimation
• Fine-grained Image-to-Image Transformation towards Visual Recognition
• Self-Reinforced Cascaded Regression for Face Alignment
• Fast Face Image Synthesis with Minimal Training
• On Improving the Generalization of Face Recognition in the Presence of Occlusions
• Collaging Class-specific GANs for Semantic Image Synthesis