Frequentist coverage and sup-norm convergence rate in Gaussian process regression

Abstract

Gaussian process (GP) regression is a powerful interpolation technique due to its flexibility in capturing non-linearity. In this paper, we provide a general framework for understanding the frequentist coverage of point-wise and simultaneous Bayesian credible sets in GP regression. As an intermediate result, we develop a Bernstein von-Mises type result under supremum norm in random design GP regression. Identifying both the mean and covariance function of the posterior distribution of the Gaussian process as regularized $M$-estimators, we show that the sampling distribution of the posterior mean function and the centered posterior distribution can be respectively approximated by two population level GPs. By developing a comparison inequality between two GPs, we provide exact characterization of frequentist coverage probabilities of Bayesian point-wise credible intervals and simultaneous credible bands of the regression function. Our results show that inference based on GP regression tends to be conservative; when the prior is under-smoothed, the resulting credible intervals and bands have minimax-optimal sizes, with their frequentist coverage converging to a non-degenerate value between their nominal level and one. As a byproduct of our theory, we show that the GP regression also yields minimax-optimal posterior contraction rate relative to the supremum norm, which provides a positive evidence to the long standing problem on optimal supremum norm contraction rate in GP regression.

References in corpus (10)

• Rates of contraction of posterior distributions based on Gaussian process priors
• Bayesian inverse problems with Gaussian priors
• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Frequentist coverage of adaptive nonparametric Bayesian credible sets
• On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures
• On Bayesian supremum norm contraction rates
• Adaptive Bernstein-von Mises theorems in Gaussian white noise
• Anisotropic function estimation using multi-bandwidth Gaussian processes
• Bayesian model selection consistency and oracle inequality with intractable marginal likelihood
• Non-asymptotic theory for nonparametric testing

Cited by in corpus (15)

• Personalized Optimization with User's Feedback
• A Divide-and-Conquer Bayesian Approach to Large-Scale Kriging
• On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces
• Distributed Bayesian Varying Coefficient Modeling Using a Gaussian Process Prior
• On the Estimation of Derivatives Using Plug-in Kernel Ridge Regression Estimators
• Learning particle swarming models from data with Gaussian processes
• Adaptive Bayesian nonparametric regression using kernel mixture of polynomials with application to partial linear model
• Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals
• Semiparametric Bayesian Inference for Local Extrema of Functions in the Presence of Noise
• Fast Statistical Leverage Score Approximation in Kernel Ridge Regression
• Uncertainty quantification for distributed regression
• Posterior Convergence of Gaussian and General Stochastic Process Regression Under Possible Misspecifications
• Quasi-Bayesian Dual Instrumental Variable Regression
• A Theoretical Framework for Bayesian Nonparametric Regression
• Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regression