Rates of contraction of posterior distributions based on Gaussian process priors

Abstract

We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.

Published in at http://dx.doi.org/10.1214/009053607000000613 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (6)

• Convergence rates of posterior distributions for noniid observations
• Reproducing kernel Hilbert spaces of Gaussian priors
• Misspecification in infinite-dimensional Bayesian statistics
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Posterior consistency of Gaussian process prior for nonparametric binary regression
• Krein's spectral theory and the Paley-Wiener expansion for fractional Brownian motion

Cited by in corpus (52)

• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Reproducing kernel Hilbert spaces of Gaussian priors
• On adaptive inference and confidence bands
• Rates of contraction for posterior distributions in $\bolds{L^r}$-metrics, $\bolds{1\le r\le\infty}$
• Lower bounds for posterior rates with Gaussian process priors
• Remarks on consistency of posterior distributions
• Frequentist coverage and sup-norm convergence rate in Gaussian process regression
• Adaptive nonparametric Bayesian inference using location-scale mixture priors
• Large Scale Variable Fidelity Surrogate Modeling
• Consistent nonparametric Bayesian inference for discretely observed scalar diffusions
• Bayes factor consistency
• Nonparametric Bayesian inference for reversible multi-dimensional diffusions
• Probabilistic forecast of nonlinear dynamical systems with uncertainty quantification
• Optimality of Poisson processes intensity learning with Gaussian processes
• Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior
• On Random Subsampling of Gaussian Process Regression: A Graphon-Based Analysis
• On polynomial-time computation of high-dimensional posterior measures by Langevin-type algorithms
• Additive Gaussian Process Regression
• A Fast and Calibrated Computer Model Emulator: An Empirical Bayes Approach
• Adaptive Gaussian Predictive Process Approximation
• General framework for projection structures
• Heavy-Tailed Density Estimation
• Fast learning rate of deep learning via a kernel perspective
• Semiparametric Bernstein-von Mises Theorem: Second Order Studies
• Unlabeled Data Help in Graph-Based Semi-Supervised Learning: A Bayesian Nonparametrics Perspective
• Designing truncated priors for direct and inverse Bayesian problems
• Graph Based Gaussian Processes on Restricted Domains
• Finite Element Representations of Gaussian Processes: Balancing Numerical and Statistical Accuracy
• Extrinsic Gaussian processes for regression and classification on manifolds
• Bayesian Trend Filtering
• Semiparametric inference using fractional posteriors
• Adaptive Bayesian procedures using random series priors
• $L_2$-small ball asymptotics for Gaussian random functions: a survey
• On the cost of Bayesian posterior mean strategy for log-concave models
• Prediction based on the Kennedy-O'Hagan calibration model: asymptotic consistency and other properties
• Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals
• Exact L_2-Small Deviation Asymptotics for Some Brownian Functionals
• A general approach to small deviation via concentration of measures
• Exponential-wrapped distributions on symmetric spaces
• Gaussian Process Regression and Classification under Mathematical Constraints with Learning Guarantees
• Posterior contraction rates for constrained deep Gaussian processes in density estimation and classication
• Posterior Contraction Rates for Gaussian Cox Processes with Non-identically Distributed Data
• Posterior contraction for deep Gaussian process priors
• Spatiotemporal Besov Priors for Bayesian Inverse Problems
• Quasi-Bayesian Dual Instrumental Variable Regression
• Semi-parametric Bernstein-von Mises in Linear Inverse Problems
• On the Convergence of Bayesian Regression Models
• Statistical Guarantees for Transformation Based Models with Applications to Implicit Variational Inference
• Bayesian estimation of nonlinear Hawkes process
• A new approach to posterior contraction rates via Wasserstein dynamics
• Thomas Bayes' walk on manifolds
• BOW: Bayesian Optimization over Windows for Motion Planning in Complex Environments