Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference

Abstract

Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose "marginally augmented variational Bayes" (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove that MAVB provides a guaranteed improvement in the approximation quality at low computational cost and induces dependencies that were assumed away by the initial factorization assumptions. I apply these techniques to a study of voter behavior using a high-dimensional application of the popular approach of multilevel regression and post-stratification (MRP). Existing estimation took hours whereas the algorithms proposed run in minutes. The posterior means are well-recovered even under strong factorization assumptions. Applying MAVB further improves the approximation by partially correcting the under-estimated variance. The proposed methodology is implemented in an open source software package.

arXiv commentAccepted version available at DOI below; corrected small typos

References in corpus (10)

• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Variational Inference: A Review for Statisticians
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Automatic Differentiation Variational Inference
• General Design Bayesian Generalized Linear Mixed Models
• Yes, but Did It Work?: Evaluating Variational Inference
• Improving multilevel regression and poststratification with structured priors
• Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations
• Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes
• A Contrastive Divergence for Combining Variational Inference and MCMC

Cited by in corpus (2)

• Scalable Bayesian computation for crossed and nested hierarchical models
• Non-conjugate variational Bayes for pseudo-likelihood mixed effect models