On optimality of Bayesian testimation in the normal means problem

Abstract

We consider a problem of recovering a high-dimensional vector $\mu$ observed in white noise, where the unknown vector $\mu$ is assumed to be sparse. The objective of the paper is to develop a Bayesian formalism which gives rise to a family of $l_0$-type penalties. The penalties are associated with various choices of the prior distributions ${\pi }_n(\cdot )$ on the number of nonzero entries of $\mu$ and, hence, are easy to interpret. The resulting Bayesian estimators lead to a general thresholding rule which accommodates many of the known thresholding and model selection procedures as particular cases corresponding to specific choices of ${\pi }_n(\cdot )$. Furthermore, they achieve optimality in a rather general setting under very mild conditions on the prior. We also specify the class of priors ${\pi }_n(\cdot )$ for which the resulting estimator is adaptively optimal (in the minimax sense) for a wide range of sparse sequences and consider several examples of such priors.

Published in at http://dx.doi.org/10.1214/009053607000000226 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Variable selection using MM algorithms

Cited by in corpus (10)

• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• An empirical Bayes mixture method for effect size and false discovery rate estimation
• Bayesian optimal adaptive estimation using a sieve prior
• Mirror averaging with sparsity priors
• High-dimensional sparse classification using exponential weighting with empirical hinge loss
• A powerful test based on tapering for use in functional data analysis
• On Bayesian "testimation" and its application to wavelet thresholding
• Sparse additive regression on a regular lattice
• Estimation of a sparse group of sparse vectors