Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences

Abstract

An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density γ, with the mixing weight chosen by marginal maximum likelihood, in the hope of adapting between sparse and dense sequences. If estimation is then carried out using the posterior median, this is a random thresholding procedure. Other thresholding rules employing the same threshold can also be used. Probability bounds on the threshold chosen by the marginal maximum likelihood approach lead to overall risk bounds over classes of signal sequences of length n, allowing for sparsity of various kinds and degrees. The signal classes considered are ``nearly black'' sequences where only a proportion ηis allowed to be nonzero, and sequences with normalized \ell_p norm bounded by η, for η>0 and 0<p\le 2. Estimation error is measured by mean qth power loss, for 0<q\le 2. For all the classes considered, and for all q in (0,2], the method achieves the optimal estimation rate as n\to \infty and η\to 0 at various rates, and in this sense adapts automatically to the sparseness or otherwise of the underlying signal. In addition the risk is uniformly bounded over all signals. If the posterior mean is used as the estimator, the results still hold for q>1. Simulations show excellent performance.

Published by the Institute of Mathematical Statistics (http://www.imstat.org) in the Annals of Statistics (http://www.imstat.org/aos/) at http://dx.doi.org/10.1214/009053604000000030

References in corpus (2)

• Empirical Bayes selection of wavelet thresholds
• General empirical Bayes wavelet methods and exactly adaptive minimax estimation

Cited by in corpus (94)

• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Size, power and false discovery rates
• Empirical Bayes selection of wavelet thresholds
• Bayesian linear regression with sparse priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• General maximum likelihood empirical Bayes estimation of normal means
• High Dimensional Statistical Inference and Random Matrices
• Bayesian Variable Selection and Estimation for Group Lasso
• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Hierarchical Bayesian sparse image reconstruction with application to MRFM
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
• An empirical Bayes mixture method for effect size and false discovery rate estimation
• Posterior consistency in linear models under shrinkage priors
• Variational Bayes for high-dimensional linear regression with sparse priors
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• On optimality of Bayesian testimation in the normal means problem
• Feature selection by Higher Criticism thresholding: optimal phase diagram
• Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO
• Stochastic expansions using continuous dictionaries: Lévy adaptive regression kernels
• Conditions for Posterior Contraction in the Sparse Normal Means Problem
• A nonparametric empirical Bayes framework for large-scale multiple testing
• Model fitting in Multiple Systems Analysis for the quantification of Modern Slavery: Classical and Bayesian approaches
• General empirical Bayes wavelet methods and exactly adaptive minimax estimation
• Empirical Bayes Matrix Factorization
• Bayesian and L1 Approaches to Sparse Unsupervised Learning
• Sparse image reconstruction for molecular imaging
• A fast algorithm for maximum likelihood estimation of mixture proportions using sequential quadratic programming
• Penalized Orthogonal-Components Regression for Large p Small n Data
• Continuous Monitoring of A/B Tests without Pain: Optional Stopping in Bayesian Testing
• Nonparametric Bayesian multiple testing for longitudinal performance stratification
• On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models
• A universal procedure for aggregating estimators
• Multichannel Boxcar Deconvolution with Growing Number of Channels
• Good, great, or lucky? Screening for firms with sustained superior performance using heavy-tailed priors
• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Large-Scale Multiple Hypothesis Testing with the Normal-Beta Prime Prior
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Bayes procedures for adaptive inference in inverse problems for the white noise model
• Frequentist optimality of Bayesian wavelet shrinkage rules for Gaussian and non-Gaussian noise
• Model Selection and Multiple Testing - A Bayesian and Empirical Bayes Overview and some New Results
• Order thresholding
• Solving the Empirical Bayes Normal Means Problem with Correlated Noise
• Adapting to Unknown Sparsity by controlling the False Discovery Rate
• Empirical Bayes estimation of normal means, accounting for uncertainty in estimated standard errors
• Dirichlet-Laplace priors for optimal shrinkage
• High-dimensional sparse classification using exponential weighting with empirical hinge loss
• Block Hyper-g Priors in Bayesian Regression
• Flexible signal denoising via flexible empirical Bayes shrinkage
• Sharp regret bounds for empirical Bayes and compound decision problems
• Besov regularity of functions with sparse random wavelet coefficients
• An approach to large-scale Quasi-Bayesian inference with spike-and-slab priors
• Empirical Bayes, SURE and Sparse Normal Mean Models
• Horseshoe Regularization for Machine Learning in Complex and Deep Models
• Sharp multiple testing boundary for sparse sequences
• Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals
• Robustness against conflicting prior information in regression
• Optimal False Discovery Control of Minimax Estimator
• Bayesian wavelet de-noising with the caravan prior
• Learning from a lot: Empirical Bayes in high-dimensional prediction settings
• On coverage and local radial rates of DDM-credible sets
• Empirical Bayes cumulative $\ell$-value multiple testing procedure for sparse sequences
• Sparsity Pattern Recovery in Bernoulli-Gaussian Signal Model
• Empirical Bayes Estimates for a 2-Way Cross-Classified Additive Model
• Mixed Effect Modeling and Variable Selection for Quantile Regression
• The Most Informative Order Statistic and its Application to Image Denoising
• Two-component Mixture Model in the Presence of Covariates
• Minimax Predictive Density for Sparse Count Data
• Network inference and community detection, based on covariance matrices, correlations and test statistics from arbitrary distributions
• Adaptive Threshold Estimation by FDR
• Unbiased Estimation of a Sparse Vector in White Gaussian Noise
• Ambiguity Sparse Processes
• Different thresholding methods on Nearest Shrunken Centroid algorithm
• ebnm: An R Package for Solving the Empirical Bayes Normal Means Problem Using a Variety of Prior Families
• What needles do sparse neural networks find in nonlinear haystacks
• Semiparametric Wavelet-based JPEG IV Estimator for endogenously truncated data
• A Nonparametric Bayesian Approach for Sparse Sequence Estimation
• On the within-family Kullback-Leibler risk in Gaussian Predictive models
• Smooth blockwise iterative thresholding: a smooth fixed point estimator based on the likelihood's block gradient
• Second Order Calibration: A Simple Way to Get Approximate Posteriors
• Posterior Contraction rate and Asymptotic Bayes Optimality for one-group shrinkage priors in sparse normal means problem
• Asymptotic Minimaxity, Optimal Posterior Concentration and Asymptotic Bayes Optimality of Horseshoe-type Priors Under Sparsity
• Efficient functional ANOVA through wavelet-domain Markov groves
• Missing Cone Artifacts Removal in ODT using Unsupervised Deep Learning in Projection Domain
• Space Alternating Penalized Kullback Proximal Point Algorithms for Maximizing Likelihood with Nondifferentiable Penalty
• Generative Parameter Sampler For Scalable Uncertainty Quantification
• Variable Selection for High-dimensional Generalized Linear Models using an Iterated Conditional Modes/Medians Algorithm
• Nonparametric Finite Mixture Models with Possible Shape Constraints: A Cubic Newton Approach
• Asymptotically optimal inference in sparse sequence models with a simple data-dependent measure
• Bayesian Edge Regression in Undirected Graphical Models to Characterize Interpatient Heterogeneity in Cancer
• Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective
• Fast Markov chain Monte Carlo for high dimensional Bayesian regression models with shrinkage priors