General framework for projection structures

Abstract

In the first part, we develop a general framework for projection structures and study several inference problems within this framework. We propose procedures based on data dependent measures (DDM) and make connections with empirical Bayes and penalization methods. The main inference problem is the uncertainty quantification (UQ), but on the way we solve the estimation, DDM-contraction problems, and a weak version of the structure recovery problem. The approach is local in that the quality of the inference procedures is measured by the local quantity, the oracle rate, which is the best trade-off between the approximation error by a projection structure and the complexity of that approximating projection structure. Like in statistical learning settings, we develop distribution-free theory as no particular model is imposed, we only assume certain mild condition on the stochastic part of the projection predictor. We introduce the excessive bias restriction (EBR) under which we establish the local confidence optimality of the constructed confidence ball. The proposed general framework unifies a very broad class of high-dimensional models and structures, interesting and important on their own right. In the second part, we apply the developed theory and demonstrate how the general results deliver a whole avenue of local and global minimax results (many new ones, some known results from the literature are improved) for particular models and structures as consequences, including white noise model and density estimation with smoothness structure, linear regression and dictionary learning with sparsity structures, biclustering and stochastic block models with clustering structure, covariance matrix estimation with banding and sparsity structures, and many others. Various adaptive minimax results over various scales follow also from our local results.

89 pages

References in corpus (12)

• Rates of contraction of posterior distributions based on Gaussian process priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Rate-optimal graphon estimation
• On risk bounds in isotonic and other shape restricted regression problems
• Frequentist coverage of adaptive nonparametric Bayesian credible sets
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Exponential Screening and optimal rates of sparse estimation
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Optimal Estimation and Completion of Matrices with Biclustering Structures
• A General Framework for Bayes Structured Linear Models
• Oracle posterior contraction rates under hierarchical priors

Cited by in corpus (2)

• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective