Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression

Abstract

We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given in closed form up to the proportionality constant. The approximation can be any distribution in the exponential family or any mixture of such distributions, which means that it can be made arbitrarily precise. Several examples illustrate the speed and accuracy of our approximation method in practice.

References in corpus (2)

• Variational Bayesian Inference with Stochastic Search
• Dynamic Trees: A Structured Variational Method Giving Efficient Propagation Rules

Cited by in corpus (12)

• Variational Bayes with Synthetic Likelihood
• Gaussian variational approximation with sparse precision matrices
• Bayesian Estimation of Mixed Multinomial Logit Models: Advances and Simulation-Based Evaluations
• Stochastic variational inference for large-scale discrete choice models using adaptive batch sizes
• Bayesian Inference for Regression Copulas
• Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models
• A stochastic variational framework for fitting and diagnosing generalized linear mixed models
• Variational inference based on a subclass of closed skew normals
• Partially factorized variational inference for high-dimensional mixed models
• Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation
• Mixed Variational Inference
• Tightening Bounds for Variational Inference by Revisiting Perturbation Theory