Minimax-optimal nonparametric regression in high dimensions

Abstract

Minimax $L_2$ risks for high-dimensional nonparametric regression are derived under two sparsity assumptions: (1) the true regression surface is a sparse function that depends only on $d=O(\log n)$ important predictors among a list of $p$ predictors, with $\log p=o(n)$; (2) the true regression surface depends on $O(n)$ predictors but is an additive function where each additive component is sparse but may contain two or more interacting predictors and may have a smoothness level different from other components. For either modeling assumption, a practicable extension of the widely used Bayesian Gaussian process regression method is shown to adaptively attain the optimal minimax rate (up to $\log n$ terms) asymptotically as both $n,p\to \infty$ with $\log p=o(n)$.

Published at http://dx.doi.org/10.1214/14-AOS1289 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (9)

• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• Rates of contraction of posterior distributions based on Gaussian process priors
• Convergence rates of posterior distributions for noniid observations
• Sparsity in multiple kernel learning
• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Reproducing kernel Hilbert spaces of Gaussian priors
• Local polynomial regression on unknown manifolds
• Rodeo: Sparse, greedy nonparametric regression

Cited by in corpus (4)

• Sensitivity analysis via the proportion of unmeasured confounding
• Incremental Intervention Effects in Studies with Dropout and Many Timepoints
• Adaptive variational Bayes: Optimality, computation and applications
• Characterizing the minimax rate of nonparametric regression under bounded star-shaped constraints