Reinterpreting Importance-Weighted Autoencoders

Abstract

The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribution. We formally derive this result, present a tighter lower bound, and visualize the implicit importance-weighted distribution.

ICLR 2017 Workshop

Cited by in corpus (32)

• An Introduction to Variational Autoencoders
• Inference Suboptimality in Variational Autoencoders
• Metrics for Deep Generative Models
• Variational Sequential Monte Carlo
• A Deep Learning Algorithm for High-Dimensional Exploratory Item Factor Analysis
• A Tutorial on Deep Latent Variable Models of Natural Language
• Advances in Variational Inference
• Bayesian Neural Ordinary Differential Equations
• A likelihood approach to nonparametric estimation of a singular distribution using deep generative models
• Streaming Variational Monte Carlo
• Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems
• Learning Hierarchical Priors in VAEs
• Learning Deep Generative Models with Annealed Importance Sampling
• Importance Weighted Adversarial Variational Autoencoders for Spike Inference from Calcium Imaging Data
• An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulations
• Failure Modes of Variational Autoencoders and Their Effects on Downstream Tasks
• On importance-weighted autoencoders
• Learning Continuous-Time Dynamics by Stochastic Differential Networks
• Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chains
• NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform
• All in the Exponential Family: Bregman Duality in Thermodynamic Variational Inference
• Automatic Differentiation Variational Inference with Mixtures
• Energy-Inspired Models: Learning with Sampler-Induced Distributions
• MCMC Variational Inference via Uncorrected Hamiltonian Annealing
• Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization
• Density Deconvolution with Normalizing Flows
• Deeply-Learned Generalized Linear Models with Missing Data
• Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
• Note on the bias and variance of variational inference
• Gaussian Mean Field Regularizes by Limiting Learned Information
• Mutual Information Constraints for Monte-Carlo Objectives
• Variational Inference with Numerical Derivatives: variance reduction through coupling