Importance Weighting and Variational Inference

Abstract

Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying the looseness in previous work. Experiments confirm IWVI's practicality for probabilistic inference. As a second contribution, we investigate inference with elliptical distributions, which improves accuracy in low dimensions, and convergence in high dimensions.

arXiv commentNeural Information Processing Systems (NIPS) 2018

Cited by in corpus (29)

• A Framework for Interdomain and Multioutput Gaussian Processes
• Generalized Variational Inference: Three arguments for deriving new Posteriors
• Disentangling Disentanglement in Variational Autoencoders
• Decision-Making with Auto-Encoding Variational Bayes
• not-MIWAE: Deep Generative Modelling with Missing not at Random Data
• Learning Discrete Distributions by Dequantization
• A Tutorial on Sparse Gaussian Processes and Variational Inference
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models
• Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence
• Particle Smoothing Variational Objectives
• An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulations
• Variational Inference with Continuously-Indexed Normalizing Flows
• Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chains
• On importance-weighted autoencoders
• Importance Weighted Hierarchical Variational Inference
• Monte Carlo Filtering Objectives: A New Family of Variational Objectives to Learn Generative Model and Neural Adaptive Proposal for Time Series
• MCMC Variational Inference via Uncorrected Hamiltonian Annealing
• On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes
• Energy-Inspired Models: Learning with Sampler-Induced Distributions
• Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
• Stochastic Variational Inference via Upper Bound
• Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
• Doubly Adaptive Importance Sampling
• Note on the bias and variance of variational inference
• Ensemble Kalman Variational Objectives: Nonlinear Latent Trajectory Inference with A Hybrid of Variational Inference and Ensemble Kalman Filter
• Reliable Categorical Variational Inference with Mixture of Discrete Normalizing Flows
• Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization
• Modulating Scalable Gaussian Processes for Expressive Statistical Learning
• Variational Inference with Numerical Derivatives: variance reduction through coupling