Invertible Conditional GANs for image editing

Abstract

Generative Adversarial Networks (GANs) have recently demonstrated to successfully approximate complex data distributions. A relevant extension of this model is conditional GANs (cGANs), where the introduction of external information allows to determine specific representations of the generated images. In this work, we evaluate encoders to inverse the mapping of a cGAN, i.e., mapping a real image into a latent space and a conditional representation. This allows, for example, to reconstruct and modify real images of faces conditioning on arbitrary attributes. Additionally, we evaluate the design of cGANs. The combination of an encoder with a cGAN, which we call Invertible cGAN (IcGAN), enables to re-generate real images with deterministic complex modifications.

Accepted paper at NIPS 2016 Workshop on Adversarial Training

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Improved Techniques for Training GANs
• Adversarial Feature Learning
• Autoencoding beyond pixels using a learned similarity metric

Cited by in corpus (148)

• cGANs with Projection Discriminator
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Fader Networks: Manipulating Images by Sliding Attributes
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Multimodal Generative Models for Scalable Weakly-Supervised Learning
• Swapping Autoencoder for Deep Image Manipulation
• Face Recognition: From Traditional to Deep Learning Methods
• MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
• Triangle Generative Adversarial Networks
• Interpreting the Latent Space of GANs for Semantic Face Editing
• DNA-GAN: Learning Disentangled Representations from Multi-Attribute Images
• Generative OpenMax for Multi-Class Open Set Classification
• Detecting GAN generated Fake Images using Co-occurrence Matrices
• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data
• RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes
• Understanding Unequal Gender Classification Accuracy from Face Images
• Modern Machine Learning Tools for Monitoring and Control of Industrial Processes: A Survey
• Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions
• GANimation: Anatomically-aware Facial Animation from a Single Image
• Semi-Latent GAN: Learning to generate and modify facial images from attributes
• MAGAN: Aligning Biological Manifolds
• Mask-aware Photorealistic Face Attribute Manipulation
• ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes
• Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks
• VITON: An Image-based Virtual Try-on Network
• JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets
• Semi-supervised Image Attribute Editing using Generative Adversarial Networks
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• Transferring GANs: generating images from limited data
• STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing
• Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
• Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
• Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation
• Robust Cross-View Gait Recognition with Evidence: A Discriminant Gait GAN (DiGGAN) Approach
• Ranking CGANs: Subjective Control over Semantic Image Attributes
• SARGAN: Spatial Attention-based Residuals for Facial Expression Manipulation
• PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editing
• Dual Swap Disentangling
• Facelet-Bank for Fast Portrait Manipulation
• What Will Your Child Look Like? DNA-Net: Age and Gender Aware Kin Face Synthesizer
• ACE: Adapting to Changing Environments for Semantic Segmentation
• UGAN: Untraceable GAN for Multi-Domain Face Translation
• 3D Brain and Heart Volume Generative Models: A Survey
• Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network
• See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar
• Survey2Survey: A deep learning generative model approach for cross-survey image mapping
• ChildPredictor: A Child Face Prediction Framework with Disentangled Learning
• Image Processing Using Multi-Code GAN Prior
• Latent Space Factorisation and Manipulation via Matrix Subspace Projection
• MulGAN: Facial Attribute Editing by Exemplar
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
• Generative Feature Replay For Class-Incremental Learning
• Detection, Attribution and Localization of GAN Generated Images
• Editing in Style: Uncovering the Local Semantics of GANs
• Multiple-Identity Image Attacks Against Face-based Identity Verification
• Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation
• Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation
• In-Domain GAN Inversion for Real Image Editing
• Learning Compositional Visual Concepts with Mutual Consistency
• Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
• GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
• Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders
• Fully-Featured Attribute Transfer
• MammoGANesis: Controlled Generation of High-Resolution Mammograms for Radiology Education
• Instance-level Facial Attributes Transfer with Geometry-Aware Flow
• SDIT: Scalable and Diverse Cross-domain Image Translation
• MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network
• Multimodal Generative Models for Compositional Representation Learning
• A Large-scale Attribute Dataset for Zero-shot Learning
• Conditional Image Generation and Manipulation for User-Specified Content
• Learning Inverse Mapping by Autoencoder based Generative Adversarial Nets
• A Survey of Deep Facial Attribute Analysis
• Coconditional Autoencoding Adversarial Networks for Chinese Font Feature Learning
• MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network
• Biphasic Learning of GANs for High-Resolution Image-to-Image Translation
• Texture Deformation Based Generative Adversarial Networks for Face Editing
• DualDis: Dual-Branch Disentangling with Adversarial Learning
• Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
• Garment Design with Generative Adversarial Networks
• Disentanglement for Discriminative Visual Recognition
• MHVAE: a Human-Inspired Deep Hierarchical Generative Model for Multimodal Representation Learning
• In&Out : Diverse Image Outpainting via GAN Inversion
• MOGAN: Morphologic-structure-aware Generative Learning from a Single Image
• Taking Control of Intra-class Variation in Conditional GANs Under Weak Supervision
• Variational Capsules for Image Analysis and Synthesis
• Unpaired Multi-Domain Image Generation via Regularized Conditional GANs
• Cycle-Consistent Inverse GAN for Text-to-Image Synthesis
• Co-supervised learning paradigm with conditional generative adversarial networks for sample-efficient classification
• GAN-Based Facial Attractiveness Enhancement
• On generating parametrised structural data using conditional generative adversarial networks
• Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation
• PriorGAN: Real Data Prior for Generative Adversarial Nets
• IntersectGAN: Learning Domain Intersection for Generating Images with Multiple Attributes
• Emotion Generation and Recognition: A StarGAN Approach
• Class-Distinct and Class-Mutual Image Generation with GANs
• Password-conditioned Anonymization and Deanonymization with Face Identity Transformers
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling
• Guided-GAN: Adversarial Representation Learning for Activity Recognition with Wearables
• Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN
• Mix and match networks: cross-modal alignment for zero-pair image-to-image translation
• Typeface Completion with Generative Adversarial Networks
• Show, Attend and Translate: Unpaired Multi-Domain Image-to-Image Translation with Visual Attention
• InfoScrub: Towards Attribute Privacy by Targeted Obfuscation
• Omni-GAN: On the Secrets of cGANs and Beyond
• Learning Controllable Disentangled Representations with Decorrelation Regularization
• How to make a pizza: Learning a compositional layer-based GAN model
• MU-GAN: Facial Attribute Editing based on Multi-attention Mechanism
• Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
• Defo-Net: Learning Body Deformation using Generative Adversarial Networks
• Flipped-Adversarial AutoEncoders
• Robust Training Using Natural Transformation
• Conditional Adversarial Camera Model Anonymization
• Face Attribute Invertion
• Fashion-AttGAN: Attribute-Aware Fashion Editing with Multi-Objective GAN
• Reconstructing faces from voices
• Coarse-to-Fine Gaze Redirection with Numerical and Pictorial Guidance
• Generative Adversarial Networks for Multimodal Representation Learning in Video Hyperlinking
• One Shot Face Swapping on Megapixels
• ClsGAN: Selective Attribute Editing Model Based On Classification Adversarial Network
• TransferI2I: Transfer Learning for Image-to-Image Translation from Small Datasets
• Gated SwitchGAN for multi-domain facial image translation
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• Force-in-domain GAN inversion
• From Continuity to Editability: Inverting GANs with Consecutive Images
• LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network
• AE-StyleGAN: Improved Training of Style-Based Auto-Encoders
• An Empirical Study on GANs with Margin Cosine Loss and Relativistic Discriminator
• CooGAN: A Memory-Efficient Framework for High-Resolution Facial Attribute Editing
• Sym-parameterized Dynamic Inference for Mixed-Domain Image Translation
• MakeupBag: Disentangling Makeup Extraction and Application
• DeFLOCNet: Deep Image Editing via Flexible Low-level Controls
• Matched sample selection with GANs for mitigating attribute confounding
• Editable Generative Adversarial Networks: Generating and Editing Faces Simultaneously
• Disassembling Object Representations without Labels
• High Diversity Attribute Guided Face Generation with GANs
• Nested Scale Editing for Conditional Image Synthesis
• Inducing Optimal Attribute Representations for Conditional GANs
• GL-GAN: Adaptive Global and Local Bilevel Optimization model of Image Generation
• Confidence-aware Adversarial Learning for Self-supervised Semantic Matching
• Learning more expressive joint distributions in multimodal variational methods
• 3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning
• Adversarial Code Learning for Image Generation
• Multi-component Image Translation for Deep Domain Generalization
• Learn to synthesize and synthesize to learn
• Expression Conditional GAN for Facial Expression-to-Expression Translation