General maximum likelihood empirical Bayes estimation of normal means

Abstract

We propose a general maximum likelihood empirical Bayes (GMLEB) method for the estimation of a mean vector based on observations with i.i.d. normal errors. We prove that under mild moment conditions on the unknown means, the average mean squared error (MSE) of the GMLEB is within an infinitesimal fraction of the minimum average MSE among all separable estimators which use a single deterministic estimating function on individual observations, provided that the risk is of greater order than $(\log n{)}^5/n$. We also prove that the GMLEB is uniformly approximately minimax in regular and weak ${\ell }_p$ balls when the order of the length-normalized norm of the unknown means is between $(\log n{)}^{{\kappa }_1}/n^{1/(p\wedge 2)}$ and $n/(\log n{)}^{{\kappa }_2}$. Simulation experiments demonstrate that the GMLEB outperforms the James--Stein and several state-of-the-art threshold estimators in a wide range of settings without much down side.

Published in at http://dx.doi.org/10.1214/08-AOS638 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
• General empirical Bayes wavelet methods and exactly adaptive minimax estimation
• Empirical Bayes methods for controlling the false discovery rate with dependent data

Cited by in corpus (55)

• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Two Modeling Strategies for Empirical Bayes Estimation
• Robust Empirical Bayes Confidence Intervals
• Optimal shrinkage estimation of mean parameters in family of distributions with quadratic variance
• Model exploration in gravitational-wave astronomy with the maximum population likelihood
• A fast algorithm for maximum likelihood estimation of mixture proportions using sequential quadratic programming
• Hypothesis testing with e-values
• Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models
• Nonparametric Empirical Bayes Estimation on Heterogeneous Data
• Deconvolution for an atomic distribution: rates of convergence
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Solving the Empirical Bayes Normal Means Problem with Correlated Noise
• On Estimating Many Means, Selection Bias, and the Bootstrap
• Noise-Induced Randomization in Regression Discontinuity Designs
• Nonparametric empirical Bayes and maximum likelihood estimation for high-dimensional data analysis
• A Bayesian Bootstrap for Mixture Models
• On nonparametric estimation of a mixing density via the predictive recursion algorithm
• Sharp regret bounds for empirical Bayes and compound decision problems
• Empirical Bayes, SURE and Sparse Normal Mean Models
• The Risk of Machine Learning
• Optimal Stopping and Worker Selection in Crowdsourcing: an Adaptive Sequential Probability Ratio Test Framework
• Integrative genetic risk prediction using nonparametric empirical Bayes classification
• Group-Linear Empirical Bayes Estimates for a Heteroscedastic Normal Mean
• The Poisson Compound Decision Problem Revisited
• Approximate nonparametric maximum likelihood inference for mixture models via convex optimization
• A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family
• Irrational Exuberance: Correcting Bias in Probability Estimates
• Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals
• A Compound Decision Approach to Covariance Matrix Estimation
• On the Minimum Attainable Risk in Permutation Invariant Problems
• Empirical Bayes PCA in high dimensions
• Asymptotic mutual information in quadratic estimation problems over compact groups
• On F-Modelling based Empirical Bayes Estimation of Variances
• Adaptive Threshold Estimation by FDR
• Multivariate, Heteroscedastic Empirical Bayes via Nonparametric Maximum Likelihood
• The Efficiency of Density Deconvolution
• Deconvolution with application to estimation of sampling probabilities and the Horvitz-Thompson estimator
• Estimating the Mean and Variance of a High-dimensional Normal Distribution Using a Mixture Prior
• A Nonparametric Maximum Likelihood Approach to Mixture of Regression
• Nonparametric Finite Mixture Models with Possible Shape Constraints: A Cubic Newton Approach
• ebnm: An R Package for Solving the Empirical Bayes Normal Means Problem Using a Variety of Prior Families
• Large-Scale Shrinkage Estimation under Markovian Dependence
• An Empirical Bayes Method for Chi-Squared Data
• Posterior Contraction rate and Asymptotic Bayes Optimality for one-group shrinkage priors in sparse normal means problem
• A Nonparametric Bayesian Approach for Sparse Sequence Estimation
• Compound decision in the presence of proxies with an application to spatio-temporal data
• Partial exchangeability of the prior via shuffling
• Asymptotic Minimaxity, Optimal Posterior Concentration and Asymptotic Bayes Optimality of Horseshoe-type Priors Under Sparsity
• A deconvolution path for mixtures
• Asymptotically optimal nonparametric empirical Bayes via predictive recursion
• Generalized maximum likelihood estimation of the mean of parameters of mixtures, with applications to sampling