Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior

Abstract

In Bayesian nonparametric models, Gaussian processes provide a popular prior choice for regression function estimation. Existing literature on the theoretical investigation of the resulting posterior distribution almost exclusively assume a fixed design for covariates. The only random design result we are aware of (van der Vaart & van Zanten, 2011) assumes the assigned Gaussian process to be supported on the smoothness class specified by the true function with probability one. This is a fairly restrictive assumption as it essentially rules out the Gaussian process prior with a squared exponential kernel when modeling rougher functions. In this article, we show that an appropriate rescaling of the above Gaussian process leads to a rate-optimal posterior distribution even when the covariates are independently realized from a known density on a compact set. The proofs are based on deriving sharp concentration inequalities for frequentist kernel estimators; the results might be of independent interest.

To appear in the Journal of Machine Learning Research

References in corpus (6)

• Rates of contraction of posterior distributions based on Gaussian process priors
• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Reproducing kernel Hilbert spaces of Gaussian priors
• Misspecification in infinite-dimensional Bayesian statistics
• Bernstein von Mises Theorems for Gaussian Regression with increasing number of regressors
• Anisotropic function estimation using multi-bandwidth Gaussian processes

Cited by in corpus (4)

• Bayesian Nonparametric Causal Inference: Information Rates and Learning Algorithms
• On Statistical Optimality of Variational Bayes
• Kriging prediction with isotropic Matérn correlations: Robustness and experimental design
• A Theoretical Framework for Bayesian Nonparametric Regression