MaskGAN: Towards Diverse and Interactive Facial Image Manipulation

Abstract

Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.

arXiv commentTo appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. The code, models and dataset are available at: https://github.com/switchablenorms/CelebAMask-HQ

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Invertible Conditional GANs for image editing
• Fader Networks: Manipulating Images by Sliding Attributes
• Deep Identity-aware Transfer of Facial Attributes
• Semantic Facial Expression Editing using Autoencoded Flow
• ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes
• Instance-level Facial Attributes Transfer with Geometry-Aware Flow

Cited by in corpus (18)

• HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment
• Privacy-Protection Drone Patrol System based on Face Anonymization
• Disentangled Makeup Transfer with Generative Adversarial Network
• MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
• Extended Labeled Faces in-the-Wild (ELFW): Augmenting Classes for Face Segmentation
• Cooperative Semantic Segmentation and Image Restoration in Adverse Environmental Conditions
• Reference-guided Face Component Editing
• MagDR: Mask-guided Detection and Reconstruction for Defending Deepfakes
• Towards Photo-Realistic Virtual Try-On by Adaptively Generating$\leftrightarrow$Preserving Image Content
• Controllable Image Synthesis via SegVAE
• End-to-End Face Parsing via Interlinked Convolutional Neural Networks
• Parallel compressive super-resolution imaging with wide field-of-view based on physics enhanced network
• Implicit Subspace Prior Learning for Dual-Blind Face Restoration
• AE-OT-GAN: Training GANs from data specific latent distribution
• ByeGlassesGAN: Identity Preserving Eyeglasses Removal for Face Images
• Making Robots Draw A Vivid Portrait In Two Minutes
• Translate the Facial Regions You Like Using Region-Wise Normalization
• Quality-Aware Network for Face Parsing