Empirical Bayes, SURE and Sparse Normal Mean Models

Abstract

This paper studies the sparse normal mean models under the empirical Bayes framework. We focus on the mixture priors with an atom at zero and a density component centered at a data driven location determined by maximizing the marginal likelihood or minimizing the Stein Unbiased Risk Estimate. We study the properties of the corresponding posterior median and posterior mean. In particular, the posterior median is a thresholding rule and enjoys the multi-direction shrinkage property that shrinks the observation toward either the origin or the data-driven location. The idea is extended by considering a finite mixture prior, which is flexible to model the cluster structure of the unknown means. We further generalize the results to heteroscedastic normal mean models. Specifically, we propose a semiparametric estimator which can be calculated efficiently by combining the familiar EM algorithm with the Pool-Adjacent-Violators algorithm for isotonic regression. The effectiveness of our methods is demonstrated via extensive numerical studies.

References in corpus (8)

• Nearly unbiased variable selection under minimax concave penalty
• Spike and slab variable selection: Frequentist and Bayesian strategies
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Empirical Bayes selection of wavelet thresholds
• General maximum likelihood empirical Bayes estimation of normal means
• In-season prediction of batting averages: A field test of empirical Bayes and Bayes methodologies
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Group-Linear Empirical Bayes Estimates for a Heteroscedastic Normal Mean

Cited by in corpus (1)

• Estimating the Mean and Variance of a High-dimensional Normal Distribution Using a Mixture Prior