Unbiased Implicit Variational Inference

Abstract

We develop unbiased implicit variational inference (UIVI), a method that expands the applicability of variational inference by defining an expressive variational family. UIVI considers an implicit variational distribution obtained in a hierarchical manner using a simple reparameterizable distribution whose variational parameters are defined by arbitrarily flexible deep neural networks. Unlike previous works, UIVI directly optimizes the evidence lower bound (ELBO) rather than an approximation to the ELBO. We demonstrate UIVI on several models, including Bayesian multinomial logistic regression and variational autoencoders, and show that UIVI achieves both tighter ELBO and better predictive performance than existing approaches at a similar computational cost.

9 pages, 3 figures

Cited by in corpus (16)

• Stochastic Normalizing Flows
• Adversarial $\alpha$-divergence Minimization for Bayesian Approximate Inference
• No MCMC for me: Amortized sampling for fast and stable training of energy-based models
• Latent space projection predictive inference
• Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chains
• Importance Weighted Hierarchical Variational Inference
• A prior-based approximate latent Riemannian metric
• Reliable Categorical Variational Inference with Mixture of Discrete Normalizing Flows
• Bayesian neural networks and dimensionality reduction
• Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs
• Copula-like Variational Inference
• Ensemble Kalman Variational Objectives: Nonlinear Latent Trajectory Inference with A Hybrid of Variational Inference and Ensemble Kalman Filter
• Tomographic Auto-Encoder: Unsupervised Bayesian Recovery of Corrupted Data
• Statistical Guarantees for Transformation Based Models with Applications to Implicit Variational Inference
• Efficient Semi-Implicit Variational Inference
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning