Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior

Abstract

The training of high-dimensional regression models on comparably sparse data is an important yet complicated topic, especially when there are many more model parameters than observations in the data. From a Bayesian perspective, inference in such cases can be achieved with the help of shrinkage prior distributions, at least for generalized linear models. However, real-world data usually possess multilevel structures, such as repeated measurements or natural groupings of individuals, which existing shrinkage priors are not built to deal with. We generalize and extend one of these priors, the R2D2 prior by Zhang et al. (2020), to linear multilevel models leading to what we call the R2D2M2 prior. The proposed prior enables both local and global shrinkage of the model parameters. It comes with interpretable hyperparameters, which we show to be intrinsically related to vital properties of the prior, such as rates of concentration around the origin, tail behavior, and amount of shrinkage the prior exerts. We offer guidelines on how to select the prior's hyperparameters by deriving shrinkage factors and measuring the effective number of non-zero model coefficients. Hence, the user can readily evaluate and interpret the amount of shrinkage implied by a specific choice of hyperparameters. Finally, we perform extensive experiments on simulated and real data, showing that our inference procedure for the prior is well calibrated, has desirable global and local regularization properties and enables the reliable and interpretable estimation of much more complex Bayesian multilevel models than was previously possible.

61 pages, 21 figures, 9 tables

References in corpus (4)

• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Graphical Test for Discrete Uniformity and its Applications in Goodness of Fit Evaluation and Multiple Sample Comparison
• Projection Predictive Inference for Generalized Linear and Additive Multilevel Models
• Group Heterogeneity Assessment for Multilevel Models

Cited by in corpus (8)

• Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms
• Efficient estimation and correction of selection-induced bias with order statistics
• Design and Structure Dependent Priors for Scale Parameters in Latent Gaussian Models
• Generalized Decomposition Priors on R2
• Gaussian distributional structural equation models: A framework for modeling latent heteroscedasticity
• The ARR2 prior: flexible predictive prior definition for Bayesian auto-regressions