Distribution Matching in Variational Inference

Abstract

With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose the limitations of Variational Autoencoders (VAEs), which consistently fail to learn marginal distributions in both latent and visible spaces. We show this to be a consequence of learning by matching conditional distributions, and the limitations of explicit model and posterior distributions. It is popular to consider Generative Adversarial Networks (GANs) as a means of overcoming these limitations, leading to hybrids of VAEs and GANs. We perform a large-scale evaluation of several VAE-GAN hybrids and analyze the implications of class probability estimation for learning distributions. While promising, we conclude that at present, VAE-GAN hybrids have limited applicability: they are harder to scale, evaluate, and use for inference compared to VAEs; and they do not improve over the generation quality of GANs.

References in corpus (19)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Conditional Image Synthesis With Auxiliary Classifier GANs
• On Calibration of Modern Neural Networks
• Semi-Supervised Learning with Deep Generative Models
• Density estimation using Real NVP
• Adversarially Learned Inference
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Unrolled Generative Adversarial Networks
• Disentangling factors of variation in deep representations using adversarial training
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Learning in Implicit Generative Models
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
• On distinguishability criteria for estimating generative models
• One-Shot Generalization in Deep Generative Models
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Fixing a Broken ELBO
• Adversarial examples for generative models

Cited by in corpus (30)

• An Introduction to Variational Autoencoders
• Disentangling by Factorising
• Learning on Attribute-Missing Graphs
• Lifelong Generative Modeling
• From Variational to Deterministic Autoencoders
• Do Deep Generative Models Know What They Don't Know?
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Getting a CLUE: A Method for Explaining Uncertainty Estimates
• Simple and Effective VAE Training with Calibrated Decoders
• Self-Supervised Variational Auto-Encoders
• Generative Latent Flow
• IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition
• Variational Autoencoders with Riemannian Brownian Motion Priors
• Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation
• On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder
• A Contrastive Learning Approach for Training Variational Autoencoder Priors
• Super-resolution Variational Auto-Encoders
• Variational Autoencoder with Implicit Optimal Priors
• Variational Leakage: The Role of Information Complexity in Privacy Leakage
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• Decomposed Adversarial Learned Inference
• On the Discrepancy between Density Estimation and Sequence Generation
• "Best-of-Many-Samples" Distribution Matching
• Likelihood Assignment for Out-of-Distribution Inputs in Deep Generative Models is Sensitive to Prior Distribution Choice
• Scalable Modeling of Spatiotemporal Data using the Variational Autoencoder: an Application in Glaucoma
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• To Regularize or Not To Regularize? The Bias Variance Trade-off in Regularized AEs
• Generating new pictures in complex datasets with a simple neural network
• Bayesian Distributional Policy Gradients
• Deep Dive into Semi-Supervised ELBO for Improving Classification Performance