Non-conjugate variational Bayes for pseudo-likelihood mixed effect models

Abstract

We propose a unified, yet simple to code, non-conjugate variational Bayes algorithm for posterior approximation of generic Bayesian generalized mixed effect models. Specifically, we consider regression models identified by a linear predictor, eventually transformed using a bijective link, where the prediction misfit is measured using, possibly non-differentiable, loss functions. Examples include generalized linear models, quasi-likelihood models, and robust regression. To address the limitations of non-conjugate settings, we employ an efficient message passing optimization strategy under a Gaussian variational approximation of the posterior. The resulting algorithms automatically account for non-conjugate priors and non-smooth losses, without requiring model-specific data-augmented representations. Besides the general formulation, we provide closed-form updates for popular model specifications, including quantile regression and support vector machines. Overall, theoretical and empirical results highlight the effectiveness of the proposed method, demonstrating its computational efficiency and approximation accuracy as an alternative to existing Bayesian techniques.

41 pages, 4 figures, 1 table

References in corpus (15)

• Variational Inference: A Review for Statisticians
• A General Framework for Updating Belief Distributions
• Automatic Differentiation Variational Inference
• Fast calibrated additive quantile regression
• Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
• Conditionally conjugate mean-field variational Bayes for logistic models
• Nonlinear quantile mixed models
• Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations
• Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference
• Semiparametric Regression using Variational Approximations
• Parameter Expansion and Efficient Inference
• A stochastic variational framework for fitting and diagnosing generalized linear mixed models
• Scaling up Data Augmentation MCMC via Calibration
• Partially factorized variational inference for high-dimensional mixed models
• Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation