Learning Disentangling and Fusing Networks for Face Completion Under Structured Occlusions

Abstract

Face completion aims to generate semantically new pixels for missing facial components. It is a challenging generative task due to large variations of face appearance. This paper studies generative face completion under structured occlusions. We treat the face completion and corruption as disentangling and fusing processes of clean faces and occlusions, and propose a jointly disentangling and fusing Generative Adversarial Network (DF-GAN). First, three domains are constructed, corresponding to the distributions of occluded faces, clean faces and structured occlusions. The disentangling and fusing processes are formulated as the transformations between the three domains. Then the disentangling and fusing networks are built to learn the transformations from unpaired data, where the encoder-decoder structure is adopted and allows DF-GAN to simulate structure occlusions by modifying the latent representations. Finally, the disentangling and fusing processes are unified into a dual learning framework along with an adversarial strategy. The proposed method is evaluated on Meshface verification problem. Experimental results on four Meshface databases demonstrate the effectiveness of our proposed method for the face completion under structured occlusions.

Submitted to CVPR 2018

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
• A Light CNN for Deep Face Representation with Noisy Labels
• Deep Learning Face Attributes in the Wild
• Generative Face Completion
• Semantic Image Inpainting with Deep Generative Models
• High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis