The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

Abstract

We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010) for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax ${\ell }_2$ risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.

arXiv commentThis version differs from the final published version in pagination and typographical detail; Available at http://projecteuclid.org/euclid.ejs/1418134265

References in corpus (9)

• Simultaneous analysis of Lasso and Dantzig selector
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Microarrays, Empirical Bayes and the Two-Groups Model
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• General maximum likelihood empirical Bayes estimation of normal means
• A comparison of the Benjamini-Hochberg procedure with some Bayesian rules for multiple testing
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Good, great, or lucky? Screening for firms with sustained superior performance using heavy-tailed priors

Cited by in corpus (22)

• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Comparison of Bayesian predictive methods for model selection
• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Dynamic Shrinkage Processes
• High-Dimensional Multivariate Posterior Consistency Under Global-Local Shrinkage Priors
• Conditions for Posterior Contraction in the Sparse Normal Means Problem
• Weighted Bayesian Bootstrap for Scalable Bayes
• Rate-optimal posterior contraction for sparse PCA
• The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?
• Function-on-Scalar Quantile Regression with Application to Mass Spectrometry Proteomics Data
• Fully Bayesian Classification with Heavy-tailed Priors for Selection in High-dimensional Features with Grouping Structure
• Large-Scale Multiple Hypothesis Testing with the Normal-Beta Prime Prior
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Adaptive group-regularized logistic elastic net regression
• Informative Bayesian Neural Network Priors for Weak Signals
• Bayesian wavelet de-noising with the caravan prior
• Phase transition in compressed sensing with horseshoe prior
• Generalized Decomposition Priors on R2
• Global-Local Shrinkage Priors for Asymptotic Point and Interval Estimation of Normal Means under Sparsity
• Posterior Contraction rate and Asymptotic Bayes Optimality for one-group shrinkage priors in sparse normal means problem
• Effect of global shrinkage parameter of horseshoe prior in compressed sensing