High-Dimensional Multivariate Posterior Consistency Under Global-Local Shrinkage Priors

Abstract

We consider sparse Bayesian estimation in the classical multivariate linear regression model with $p$ regressors and $q$ response variables. In univariate Bayesian linear regression with a single response $y$, shrinkage priors which can be expressed as scale mixtures of normal densities are popular for obtaining sparse estimates of the coefficients. In this paper, we extend the use of these priors to the multivariate case to estimate a $p\times q$ coefficients matrix $\mathbf{B}$. We derive sufficient conditions for posterior consistency under the Bayesian multivariate linear regression framework and prove that our method achieves posterior consistency even when $p>n$ and even when $p$ grows at nearly exponential rate with the sample size. We derive an efficient Gibbs sampling algorithm and provide the implementation in a comprehensive R package called MBSP. Finally, we demonstrate through simulations and data analysis that our model has excellent finite sample performance.

18 pages, 3 tables, 1 figure. More technical details of computation added to Section 4.2, proofs moved to separate online supplement

References in corpus (3)

• Bayesian variable selection with shrinking and diffusing priors
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart

Cited by in corpus (5)

• Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions
• The EAS approach to variable selection for multivariate response data in high-dimensional settings
• Bayesian Variable Selection for Multi-Outcome Models Through Shared Shrinkage
• Ultra High-dimensional Multivariate Posterior Contraction Rate Under Shrinkage Priors
• Posterior consistency in multi-response regression models with non-informative priors for the error covariance matrix in growing dimensions