On the Quantitative Analysis of Decoder-Based Generative Models

Abstract

The past several years have seen remarkable progress in generative models which produce convincing samples of images and other modalities. A shared component of many powerful generative models is a decoder network, a parametric deep neural net that defines a generative distribution. Examples include variational autoencoders, generative adversarial networks, and generative moment matching networks. Unfortunately, it can be difficult to quantify the performance of these models because of the intractability of log-likelihood estimation, and inspecting samples can be misleading. We propose to use Annealed Importance Sampling for evaluating log-likelihoods for decoder-based models and validate its accuracy using bidirectional Monte Carlo. The evaluation code is provided at https://github.com/tonywu95/eval_gen. Using this technique, we analyze the performance of decoder-based models, the effectiveness of existing log-likelihood estimators, the degree of overfitting, and the degree to which these models miss important modes of the data distribution.

arXiv commentAccepted to ICLR2017

Cited by in corpus (15)

• NVAE: A Deep Hierarchical Variational Autoencoder
• NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
• Adversarially Learned Anomaly Detection
• Decision-Making with Auto-Encoding Variational Bayes
• A Non-Parametric Test to Detect Data-Copying in Generative Models
• Regularized Autoencoders via Relaxed Injective Probability Flow
• Learning Deep Generative Models with Annealed Importance Sampling
• Failure Modes of Variational Autoencoders and Their Effects on Downstream Tasks
• Bidirectional Generative Modeling Using Adversarial Gradient Estimation
• Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chains
• Pretraining Image Encoders without Reconstruction via Feature Prediction Loss
• MCMC Variational Inference via Uncorrected Hamiltonian Annealing
• q-Paths: Generalizing the Geometric Annealing Path using Power Means
• Dist-GAN: An Improved GAN using Distance Constraints
• GILBO: One Metric to Measure Them All