Generalized Decomposition Priors on R2

Abstract

The adoption of continuous shrinkage priors in high-dimensional linear models has gained widespread attention due to their practical and theoretical advantages. Among them, the R2D2 prior has gained popularity for its intuitive specification of the proportion of explained variance (R2) and its theoretically grounded properties. The R2D2 prior allocates variance among regression terms through a Dirichlet decomposition. However, this approach inherently limits the dependency structure among variance components to the negative dependence modeled by the Dirichlet distribution, which is fully determined by the mean. This limitation hinders the prior's ability to capture more nuanced or positive dependency patterns that may arise in real-world data. To address this, we propose the Generalized Decomposition R2 (GDR2) prior, which replaces the Dirichlet decomposition with the more flexible Logistic-Normal distribution and its variants. By allowing richer dependency structures, the GDR2 prior accommodates more realistic and adaptable competition among variance components, enhancing the expressiveness and applicability of R2-based priors in practice. Through simulations and real-world benchmarks, we demonstrate that the GDR2 prior improves out-of-sample predictive performance and parameter recovery compared to the R2D2 prior. Our framework bridges the gap between flexibility in variance decomposition and practical implementation, advancing the utility of shrinkage priors in complex regression settings.

50 pages, 28 figures

References in corpus (10)

• A correlated topic model of Science
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Model Selection in Bayesian Neural Networks via Horseshoe Priors
• Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior
• Projection Predictive Inference for Generalized Linear and Additive Multilevel Models
• Group Inverse-Gamma Gamma Shrinkage for Sparse Regression with Block-Correlated Predictors
• Schlömilch integrals and probability distributions on the simplex