Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective
arXiv:2009.03969
Abstract
We study the convergence rates of empirical Bayes posterior distributions for nonparametric and high-dimensional inference. We show that as long as the hyperparameter set is discrete, the empirical Bayes posterior distribution induced by the maximum marginal likelihood estimator can be regarded as a variational approximation to a hierarchical Bayes posterior distribution. This connection between empirical Bayes and variational Bayes allows us to leverage the recent results in the variational Bayes literature, and directly obtains the convergence rates of empirical Bayes posterior distributions from a variational perspective. For a more general hyperparameter set that is not necessarily discrete, we introduce a new technique called "prior decomposition" to deal with prior distributions that can be written as convex combinations of probability measures whose supports are low-dimensional subspaces. This leads to generalized versions of the classical "prior mass and testing" conditions for the convergence rates of empirical Bayes. Our theory is applied to a number of statistical estimation problems including nonparametric density estimation and sparse linear regression.
References in corpus (13)
- Simultaneous analysis of Lasso and Dantzig selector
- Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
- Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
- Frequentist Consistency of Variational Bayes
- Frequentist coverage of adaptive nonparametric Bayesian credible sets
- Lower bounds on the performance of polynomial-time algorithms for sparse linear regression
- Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
- Rates of convergence for the posterior distributions of mixtures of Betas and adaptive nonparametric estimation of the density
- Optimal Estimation and Completion of Matrices with Biclustering Structures
- Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
- Data-driven priors and their posterior concentration rates
- General framework for projection structures
- Oracle posterior contraction rates under hierarchical priors