Representation Learning by Rotating Your Faces

Abstract

The large pose discrepancy between two face images is one of the fundamental challenges in automatic face recognition. Conventional approaches to pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks jointly to allow them to leverage each other. To this end, this paper proposes a Disentangled Representation learning-Generative Adversarial Network (DR-GAN) with three distinct novelties. First, the encoder-decoder structure of the generator enables DR-GAN to learn a representation that is both generative and discriminative, which can be used for face image synthesis and pose-invariant face recognition. Second, this representation is explicitly disentangled from other face variations such as pose, through the pose code provided to the decoder and pose estimation in the discriminator. Third, DR-GAN can take one or multiple images as the input, and generate one unified identity representation along with an arbitrary number of synthetic face images. Extensive quantitative and qualitative evaluation on a number of controlled and in-the-wild databases demonstrate the superiority of DR-GAN over the state of the art in both learning representations and rotating large-pose face images.

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

References in corpus (9)

• Conditional Generative Adversarial Nets
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Learning Face Representation from Scratch
• Additive Margin Softmax for Face Verification
• Deep Convolutional Inverse Graphics Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition
• On Learning 3D Face Morphable Model from In-the-wild Images
• Ways of Conditioning Generative Adversarial Networks

Cited by in corpus (34)

• The Creation and Detection of Deepfakes: A Survey
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• On Learning 3D Face Morphable Model from In-the-wild Images
• Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition
• Face Recognition: From Traditional to Deep Learning Methods
• Face Generation and Editing with StyleGAN: A Survey
• Deep Learning Based Single Sample Per Person Face Recognition: A Survey
• Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks
• OSTeC: One-Shot Texture Completion
• Face Hallucination with Finishing Touches
• On the Detection of Digital Face Manipulation
• Quantum semi-supervised generative adversarial network for enhanced data classification
• A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion
• Locality-aware Channel-wise Dropout for Occluded Face Recognition
• Nonlinear 3D Face Morphable Model
• Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images
• Gait Recognition via Disentangled Representation Learning
• Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data
• Facial UV Map Completion for Pose-invariant Face Recognition: A Novel Adversarial Approach based on Coupled Attention Residual UNets
• Cyber Vaccine for Deepfake Immunity
• Towards High-fidelity Nonlinear 3D Face Morphable Model
• On Learning Disentangled Representations for Gait Recognition
• On Disentangling Spoof Trace for Generic Face Anti-Spoofing
• BoostGAN for Occlusive Profile Face Frontalization and Recognition
• Towards Interpretable Face Recognition
• UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition
• Pipeline Generative Adversarial Networks for Facial Images Generation with Multiple Attributes
• An Assessment of GANs for Identity-related Applications
• Dual Encoder-Decoder based Generative Adversarial Networks for Disentangled Facial Representation Learning
• Adaptive Multiscale Illumination-Invariant Feature Representation for Undersampled Face Recognition
• Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction
• ReshapeGAN: Object Reshaping by Providing A Single Reference Image