Gaussian variational approximation with sparse precision matrices

Abstract

We consider the problem of learning a Gaussian variational approximation to the posterior distribution for a high-dimensional parameter, where we impose sparsity in the precision matrix to reflect appropriate conditional independence structure in the model. Incorporating sparsity in the precision matrix allows the Gaussian variational distribution to be both flexible and parsimonious, and the sparsity is achieved through parameterization in terms of the Cholesky factor. Efficient stochastic gradient methods which make appropriate use of gradient information for the target distribution are developed for the optimization. We consider alternative estimators of the stochastic gradients which have lower variation and are more stable. Our approach is illustrated using generalized linear mixed models and state space models for time series.

arXiv comment18 pages, 9 figures

References in corpus (3)

• Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
• Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations
• A stochastic variational framework for fitting and diagnosing generalized linear mixed models

Cited by in corpus (8)

• Variational Inference: A Review for Statisticians
• Variational Bayes with Synthetic Likelihood
• Variational Bayesian Approximation of Inverse Problems using Sparse Precision Matrices
• Variational inference based on a subclass of closed skew normals
• Partially factorized variational inference for high-dimensional mixed models
• Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation
• R-VGAL: A Sequential Variational Bayes Algorithm for Generalised Linear Mixed Models
• Variational inference for hierarchical models with conditional scale and skewness corrections