Perceptual Generative Autoencoders

Abstract

Modern generative models are usually designed to match target distributions directly in the data space, where the intrinsic dimension of data can be much lower than the ambient dimension. We argue that this discrepancy may contribute to the difficulties in training generative models. We therefore propose to map both the generated and target distributions to a latent space using the encoder of a standard autoencoder, and train the generator (or decoder) to match the target distribution in the latent space. Specifically, we enforce the consistency in both the data space and the latent space with theoretically justified data and latent reconstruction losses. The resulting generative model, which we call a perceptual generative autoencoder (PGA), is then trained with a maximum likelihood or variational autoencoder (VAE) objective. With maximum likelihood, PGAs generalize the idea of reversible generative models to unrestricted neural network architectures and arbitrary number of latent dimensions. When combined with VAEs, PGAs substantially improve over the baseline VAEs in terms of sample quality. Compared to other autoencoder-based generative models using simple priors, PGAs achieve state-of-the-art FID scores on CIFAR-10 and CelebA.

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• NICE: Non-linear Independent Components Estimation
• Adversarial Autoencoders
• Generalized Denoising Auto-Encoders as Generative Models
• From Variational to Deterministic Autoencoders
• On the Latent Space of Wasserstein Auto-Encoders

Cited by in corpus (7)

• Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior
• Hierarchical Quantized Autoencoders
• A Contrastive Learning Approach for Training Variational Autoencoder Priors
• Solving Inverse Problems by Joint Posterior Maximization with a VAE Prior
• A Machine Learning Enhanced Approach for Automated Sunquake Detection in Acoustic Emission Maps
• Classify and Generate: Using Classification Latent Space Representations for Image Generations
• A multiscale and multicriteria Generative Adversarial Network to synthesize 1-dimensional turbulent fields