Fader Networks: Manipulating Images by Sliding Attributes

Abstract

This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes. We present evidence that our model can significantly change the perceived value of the attributes while preserving the naturalness of images.

arXiv commentNIPS 2017

References in corpus (6)

• Deep Convolutional Inverse Graphics Network
• Invertible Conditional GANs for image editing
• Unsupervised Cross-Domain Image Generation
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Disentangling factors of variation in deep representations using adversarial training
• Learning to Pivot with Adversarial Networks

Cited by in corpus (32)

• MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
• Interpreting the Latent Space of GANs for Semantic Face Editing
• Driving-Signal Aware Full-Body Avatars
• Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning
• Disentangled Adversarial Autoencoder for Subject-Invariant Physiological Feature Extraction
• CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfer
• Mask-aware Photorealistic Face Attribute Manipulation
• 3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks
• Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI
• Coherent Point Drift Networks: Unsupervised Learning of Non-Rigid Point Set Registration
• Ranking CGANs: Subjective Control over Semantic Image Attributes
• Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement
• EEG-based Texture Roughness Classification in Active Tactile Exploration with Invariant Representation Learning Networks
• ImageEye: Batch Image Processing Using Program Synthesis
• Survey2Survey: A deep learning generative model approach for cross-survey image mapping
• Image Processing Using Multi-Code GAN Prior
• Unsupervised 3D Reconstruction from a Single Image via Adversarial Learning
• Multi-View Data Generation Without View Supervision
• Deepfake Video Forensics based on Transfer Learning
• GANHopper: Multi-Hop GAN for Unsupervised Image-to-Image Translation
• Recursive Chaining of Reversible Image-to-image Translators For Face Aging
• Human-interpretable model explainability on high-dimensional data
• A3T: Adversarially Augmented Adversarial Training
• Instance-level Facial Attributes Transfer with Geometry-Aware Flow
• DeepBlur: A Simple and Effective Method for Natural Image Obfuscation
• CRAFT: Complementary Recommendations Using Adversarial Feature Transformer
• VAE/WGAN-Based Image Representation Learning For Pose-Preserving Seamless Identity Replacement In Facial Images
• Dual-reference Age Synthesis
• Learning Invariances for Policy Generalization
• Face Aging via Diffusion-based Editing
• TranSTYLer: Multimodal Behavioral Style Transfer for Facial and Body Gestures Generation
• DEFT: Distilling Entangled Factors by Preventing Information Diffusion