Optimizing the Latent Space of Generative Networks

Abstract

Generative Adversarial Networks (GANs) have achieved remarkable results in the task of generating realistic natural images. In most successful applications, GAN models share two common aspects: solving a challenging saddle point optimization problem, interpreted as an adversarial game between a generator and a discriminator functions; and parameterizing the generator and the discriminator as deep convolutional neural networks. The goal of this paper is to disentangle the contribution of these two factors to the success of GANs. In particular, we introduce Generative Latent Optimization (GLO), a framework to train deep convolutional generators using simple reconstruction losses. Throughout a variety of experiments, we show that GLO enjoys many of the desirable properties of GANs: synthesizing visually-appealing samples, interpolating meaningfully between samples, and performing linear arithmetic with noise vectors; all of this without the adversarial optimization scheme.

References in corpus (7)

• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Understanding deep learning requires rethinking generalization
• Energy-based Generative Adversarial Network
• Unsupervised Learning by Predicting Noise
• Precise Recovery of Latent Vectors from Generative Adversarial Networks
• Inverting The Generator Of A Generative Adversarial Network (II)

Cited by in corpus (43)

• Deep Image Prior
• Deep Clustering for Unsupervised Learning of Visual Features
• NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination
• Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures
• Graph Neural Networks Based Detection of Stealth False Data Injection Attacks in Smart Grids
• 2022 Review of Data-Driven Plasma Science
• Interpreting the Latent Space of GANs for Semantic Face Editing
• A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution
• ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation
• Robust in Practice: Adversarial Attacks on Quantum Machine Learning
• Generative Models for Low-Rank Video Representation and Reconstruction
• BourGAN: Generative Networks with Metric Embeddings
• Approximability of Discriminators Implies Diversity in GANs
• Gaussian Material Synthesis
• Context-aware Synthesis for Video Frame Interpolation
• SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks
• A Classification Supervised Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids
• A Survey on Deep Generative 3D-aware Image Synthesis
• Generative adversarial network for super-resolution imaging through a fiber
• Generative Latent Flow
• Non-Adversarial Unsupervised Word Translation
• Prediction Under Uncertainty with Error-Encoding Networks
• Analyzing drop coalescence in microfluidic device with a deep learning generative model
• ShapeFlow: Learnable Deformations Among 3D Shapes
• Enhancing Deformable Convolution based Video Frame Interpolation with Coarse-to-fine 3D CNN
• Guided Variational Autoencoder for Disentanglement Learning
• Geometric Entropic Exploration
• Continuous Mixtures of Tractable Probabilistic Models
• Enhanced Quadratic Video Interpolation
• Neural Fields in Visual Computing and Beyond
• Learning Latent Space Energy-Based Prior Model
• Semantic Interpolation in Implicit Models
• Definition-independent Formalization of Soundscapes: Towards a Formal Methodology
• Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation
• Generative Visual Rationales
• Stingray Detection of Aerial Images Using Augmented Training Images Generated by A Conditional Generative Model
• Residual-Recursion Autoencoder for Shape Illustration Images
• An Iterative Closest Points Approach to Neural Generative Models
• TunaGAN: Interpretable GAN for Smart Editing
• Reducing the Representation Error of GAN Image Priors Using the Deep Decoder
• Analytical Probability Distributions and EM-Learning for Deep Generative Networks
• Improving Visual Recognition using Ambient Sound for Supervision
• Multiclass non-Adversarial Image Synthesis, with Application to Classification from Very Small Sample