Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences

Abstract

We consider full Bayesian inference in the multivariate normal mean model in the situation that the mean vector is sparse. The prior distribution on the vector of means is constructed hierarchically by first choosing a collection of nonzero means and next a prior on the nonzero values. We consider the posterior distribution in the frequentist set-up that the observations are generated according to a fixed mean vector, and are interested in the posterior distribution of the number of nonzero components and the contraction of the posterior distribution to the true mean vector. We find various combinations of priors on the number of nonzero coefficients and on these coefficients that give desirable performance. We also find priors that give suboptimal convergence, for instance, Gaussian priors on the nonzero coefficients. We illustrate the results by simulations.

Published in at http://dx.doi.org/10.1214/12-AOS1029 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (12)

• Simultaneous analysis of Lasso and Dantzig selector
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Empirical Bayes selection of wavelet thresholds
• General maximum likelihood empirical Bayes estimation of normal means
• Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
• Estimation and confidence sets for sparse normal mixtures
• On optimality of Bayesian testimation in the normal means problem
• Lower bounds for posterior rates with Gaussian process priors
• General empirical Bayes wavelet methods and exactly adaptive minimax estimation

Cited by in corpus (73)

• Bayesian linear regression with sparse priors
• Bayesian Variable Selection and Estimation for Group Lasso
• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Posterior contraction in sparse Bayesian factor models for massive covariance matrices
• Variational Bayes for high-dimensional linear regression with sparse priors
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Adaptive Bernstein-von Mises theorems in Gaussian white noise
• Nearly optimal Bayesian Shrinkage for High Dimensional Regression
• Conditions for Posterior Contraction in the Sparse Normal Means Problem
• Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings
• Penalising model component complexity: A principled, practical approach to constructing priors
• Rate-optimal posterior contraction for sparse PCA
• Gibbs posterior inference on multivariate quantiles
• Data-driven priors and their posterior concentration rates
• Minimax Optimal Bayesian Aggregation
• Nonparametric Empirical Bayes Estimation on Heterogeneous Data
• Adaptive Bayesian density regression for high-dimensional data
• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Spike and slab variational Bayes for high dimensional logistic regression
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Bayesian detection of piecewise linear trends in replicated time-series with application to growth data modelling
• Adaptive Supremum Norm Posterior Contraction: Wavelet Spike-and-Slab and Anisotropic Besov Spaces
• Bayesian Effect Selection in Structured Additive Distributional Regression Models
• Risk quantification for the thresholding rule for multiple testing using Gaussian scale mixtures
• Variational Bayes for high-dimensional proportional hazards models with applications within gene expression
• The matryoshka doll prior: principled multiplicity correction in Bayesian model comparison
• High-dimensional sparse classification using exponential weighting with empirical hinge loss
• Variational approximations of empirical Bayes posteriors in high-dimensional linear models
• Optimal Bayesian estimation in stochastic block models
• The Inverse Gamma-Gamma Prior for Optimal Posterior Contraction and Multiple Hypothesis Testing
• Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions
• Variable Selection with ABC Bayesian Forests
• Neuronized Priors for Bayesian Sparse Linear Regression
• General framework for projection structures
• An approach to large-scale Quasi-Bayesian inference with spike-and-slab priors
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Optimal Bayesian estimation of Gaussian mixtures with growing number of components
• Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithm
• Horseshoe Regularization for Machine Learning in Complex and Deep Models
• Oracle posterior contraction rates under hierarchical priors
• Learning from a lot: Empirical Bayes in high-dimensional prediction settings
• Optimal False Discovery Control of Minimax Estimator
• On the Exponentially Weighted Aggregate with the Laplace Prior
• On coverage and local radial rates of DDM-credible sets
• Bayesian high-dimensional linear regression with generic spike-and-slab priors
• Bayesian Stability Selection and Inference on Selection Probabilities
• Generalized Decomposition Priors on R2
• Bayesian Scalable Precision Factor Analysis for Massive Sparse Gaussian Graphical Models
• Mixed Effect Modeling and Variable Selection for Quantile Regression
• On the cost of Bayesian posterior mean strategy for log-concave models
• Bayesian factorizations of big sparse tensors
• A fast asynchronous MCMC sampler for sparse Bayesian inference
• Inference for multiple treatment effects using confounder importance learning
• Nonasymptotic Laplace approximation under model misspecification
• Bayesian variable selection in linear regression models with instrumental variables
• Global-Local Shrinkage Priors for Asymptotic Point and Interval Estimation of Normal Means under Sparsity
• Asymptotically optimal inference in sparse sequence models with a simple data-dependent measure
• Bayesian Edge Regression in Undirected Graphical Models to Characterize Interpatient Heterogeneity in Cancer
• Adaptive posterior convergence in sparse high dimensional clipped generalized linear models
• Bayesian Shrinkage towards Sharp Minimaxity
• Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors
• Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective
• Bayesian estimation of a sparse precision matrix
• Generative Parameter Sampler For Scalable Uncertainty Quantification
• A Nonparametric Bayesian Approach for Sparse Sequence Estimation
• A Theoretical Framework for Bayesian Nonparametric Regression
• Posterior Contraction Rate of Sparse Latent Feature Models with Application to Proteomics
• Unified Bayesian theory of sparse linear regression with nuisance parameters
• Semiparametric Functional Factor Models with Bayesian Rank Selection
• Asymptotic Minimaxity, Optimal Posterior Concentration and Asymptotic Bayes Optimality of Horseshoe-type Priors Under Sparsity