Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations

Abstract

The effects of different parametrizations on the convergence of Bayesian computational algorithms for hierarchical models are well explored. Techniques such as centering, noncentering and partial noncentering can be used to accelerate convergence in MCMC and EM algorithms but are still not well studied for variational Bayes (VB) methods. As a fast deterministic approach to posterior approximation, VB is attracting increasing interest due to its suitability for large high-dimensional data. Use of different parametrizations for VB has not only computational but also statistical implications, as different parametrizations are associated with different factorized posterior approximations. We examine the use of partially noncentered parametrizations in VB for generalized linear mixed models (GLMMs). Our paper makes four contributions. First, we show how to implement an algorithm called nonconjugate variational message passing for GLMMs. Second, we show that the partially noncentered parametrization can adapt to the quantity of information in the data and determine a parametrization close to optimal. Third, we show that partial noncentering can accelerate convergence and produce more accurate posterior approximations than centering or noncentering. Finally, we demonstrate how the variational lower bound, produced as part of the computation, can be useful for model selection.

Published in at http://dx.doi.org/10.1214/13-STS418 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Inferring Parameters and Structure of Latent Variable Models by Variational Bayes
• A General Framework for the Parametrization of Hierarchical Models
• General Design Bayesian Generalized Linear Mixed Models
• Variational approximation for mixtures of linear mixed models

Cited by in corpus (10)

• Variational Inference: A Review for Statisticians
• Gaussian variational approximation with sparse precision matrices
• Statistically-estimated tree composition for the northeastern United States at the time of Euro-American settlement
• Stochastic variational inference for large-scale discrete choice models using adaptive batch sizes
• Variational inference for sparse spectrum Gaussian process regression
• Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference
• A stochastic variational framework for fitting and diagnosing generalized linear mixed models
• Partially factorized variational inference for high-dimensional mixed models
• Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation
• Non-conjugate variational Bayes for pseudo-likelihood mixed effect models