Neural Photo Editing with Introspective Adversarial Networks

Abstract

The increasingly photorealistic sample quality of generative image models suggests their feasibility in applications beyond image generation. We present the Neural Photo Editor, an interface that leverages the power of generative neural networks to make large, semantically coherent changes to existing images. To tackle the challenge of achieving accurate reconstructions without loss of feature quality, we introduce the Introspective Adversarial Network, a novel hybridization of the VAE and GAN. Our model efficiently captures long-range dependencies through use of a computational block based on weight-shared dilated convolutions, and improves generalization performance with Orthogonal Regularization, a novel weight regularization method. We validate our contributions on CelebA, SVHN, and CIFAR-100, and produce samples and reconstructions with high visual fidelity.

arXiv comment10 pages, 7 figures, 3 tables

References in corpus (13)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Densely Connected Convolutional Networks
• Adversarial Feature Learning
• Autoencoding beyond pixels using a learned similarity metric
• Auxiliary Deep Generative Models
• Stacked What-Where Auto-encoders
• Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks
• Generating Images with Perceptual Similarity Metrics based on Deep Networks
• Generative Visual Manipulation on the Natural Image Manifold
• Discriminative Regularization for Generative Models
• An All-In-One Convolutional Neural Network for Face Analysis

Cited by in corpus (63)

• Generative Adversarial Networks: An Overview
• Spectral Normalization for Generative Adversarial Networks
• An Introduction to Variational Autoencoders
• Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss
• Adversarially Learned Inference
• FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
• Unsupervised Cross-Domain Image Generation
• Perceptual Adversarial Networks for Image-to-Image Transformation
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• An Introduction to Image Synthesis with Generative Adversarial Nets
• RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces
• A Systematic Survey of Regularization and Normalization in GANs
• Activation Maximization Generative Adversarial Nets
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• ArtGAN: Artwork Synthesis with Conditional Categorical GANs
• Facelet-Bank for Fast Portrait Manipulation
• Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees
• ImageEye: Batch Image Processing Using Program Synthesis
• Interactive Image Manipulation with Natural Language Instruction Commands
• Globally Injective ReLU Networks
• Optimal transport maps for distribution preserving operations on latent spaces of Generative Models
• Interactive Neural Painting
• Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
• Competing Against Equilibria in Zero-Sum Games with Evolving Payoffs
• Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
• Adversarial Robustness through the Lens of Convolutional Filters
• Instance-level Facial Attributes Transfer with Geometry-Aware Flow
• RankGAN: A Maximum Margin Ranking GAN for Generating Faces
• Jigsaw-VAE: Towards Balancing Features in Variational Autoencoders
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Smart, Sparse Contours to Represent and Edit Images
• Taking Control of Intra-class Variation in Conditional GANs Under Weak Supervision
• Regularizing Neural Networks via Minimizing Hyperspherical Energy
• Depth Structure Preserving Scene Image Generation
• Decomposed Adversarial Learned Inference
• On generating parametrised structural data using conditional generative adversarial networks
• Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs
• Reciprocal Adversarial Learning via Characteristic Functions
• Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation
• Approximated Orthonormal Normalisation in Training Neural Networks
• Intuitive, Interactive Beard and Hair Synthesis with Generative Models
• Ensembling with Deep Generative Views
• Semantic Image Manipulation Using Scene Graphs
• Exploiting GAN Internal Capacity for High-Quality Reconstruction of Natural Images
• Introspective Generative Modeling: Decide Discriminatively
• One-Shot Domain Adaptation For Face Generation
• Image Captions are Natural Prompts for Text-to-Image Models
• Unsupervised Representation Adversarial Learning Network: from Reconstruction to Generation
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• Inverse mapping of face GANs
• A Generative Approach Towards Improved Robotic Detection of Marine Litter
• Kernel Mean Matching for Content Addressability of GANs
• Towards Disentangling Latent Space for Unsupervised Semantic Face Editing
• One-element Batch Training by Moving Window
• CanvasGAN: A simple baseline for text to image generation by incrementally patching a canvas
• Learning with Hyperspherical Uniformity
• Adversarial Code Learning for Image Generation
• MakeupBag: Disentangling Makeup Extraction and Application
• Comparing Dynamical Models Through Diffeomorphic Vector Field Alignment
• Diversity Regularized Adversarial Learning