Generalized Decomposition Priors on R2

Abstract

The adoption of continuous shrinkage priors in high-dimensional linear models has gained widespread attention due to their practical and theoretical advantages. Among them, the R2D2 prior has gained popularity for its intuitive specification of the proportion of explained variance (R2) and its theoretically grounded properties. The R2D2 prior allocates variance among regression terms through a Dirichlet decomposition. However, this approach inherently limits the dependency structure among variance components to the negative dependence modeled by the Dirichlet distribution, which is fully determined by the mean. This limitation hinders the prior's ability to capture more nuanced or positive dependency patterns that may arise in real-world data. To address this, we propose the Generalized Decomposition R2 (GDR2) prior, which replaces the Dirichlet decomposition with the more flexible Logistic-Normal distribution and its variants. By allowing richer dependency structures, the GDR2 prior accommodates more realistic and adaptable competition among variance components, enhancing the expressiveness and applicability of R2-based priors in practice. Through simulations and real-world benchmarks, we demonstrate that the GDR2 prior improves out-of-sample predictive performance and parameter recovery compared to the R2D2 prior. Our framework bridges the gap between flexibility in variance decomposition and practical implementation, advancing the utility of shrinkage priors in complex regression settings.

arXiv comment50 pages, 28 figures

References in corpus (18)

• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Variational Inference: A Review for Statisticians
• MCMC using Hamiltonian dynamics
• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Rank-normalization, folding, and localization: An improved $\hat{R}$ for assessing convergence of MCMC
• A correlated topic model of Science
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Model Selection in Bayesian Neural Networks via Horseshoe Priors
• Conditions for Posterior Contraction in the Sparse Normal Means Problem
• Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior
• Projection Predictive Inference for Generalized Linear and Additive Multilevel Models
• Large-Scale Multiple Hypothesis Testing with the Normal-Beta Prime Prior
• Group Inverse-Gamma Gamma Shrinkage for Sparse Regression with Block-Correlated Predictors
• Schlömilch integrals and probability distributions on the simplex