Rates of contraction of posterior distributions based on Gaussian process priors

Abstract

We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.

arXiv commentPublished in at http://dx.doi.org/10.1214/009053607000000613 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (6)

• Convergence rates of posterior distributions for noniid observations
• Reproducing kernel Hilbert spaces of Gaussian priors
• Misspecification in infinite-dimensional Bayesian statistics
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Posterior consistency of Gaussian process prior for nonparametric binary regression
• Krein's spectral theory and the Paley-Wiener expansion for fractional Brownian motion

Cited by in corpus (125)

• Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth
• Reproducing kernel Hilbert spaces of Gaussian priors
• On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures
• Nonparametric Bernstein-von Mises theorems in Gaussian white noise
• Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems
• On adaptive inference and confidence bands
• The semiparametric Bernstein-von Mises theorem
• On Bayesian supremum norm contraction rates
• Nonparametric Bayesian posterior contraction rates for discretely observed scalar diffusions
• A Bernstein-von Mises theorem for smooth functionals in semiparametric models
• Bayesian inverse problems with non-conjugate priors
• Rates of contraction for posterior distributions in $\bolds{L^r}$-metrics, $\bolds{1\le r\le\infty}$
• Posterior Consistency for Bayesian Inverse Problems through Stability and Regression Results
• Minimax-optimal nonparametric regression in high dimensions
• Anisotropic function estimation using multi-bandwidth Gaussian processes
• Lower bounds for posterior rates with Gaussian process priors
• On the Brittleness of Bayesian Inference
• Bayesian Nonparametric Weighted Sampling Inference
• Posterior convergence for approximated unknowns in non-Gaussian statistical inverse problems
• Remarks on consistency of posterior distributions
• Bayesian optimal adaptive estimation using a sieve prior
• Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions
• Frequentist coverage and sup-norm convergence rate in Gaussian process regression
• Adaptive nonparametric Bayesian inference using location-scale mixture priors
• Large Scale Variable Fidelity Surrogate Modeling
• A Nonparametric Bayesian Methodology for Regression Discontinuity Designs
• Consistent nonparametric Bayesian inference for discretely observed scalar diffusions
• Personalized Optimization with User's Feedback
• Gaussian process methods for one-dimensional diffusions: optimal rates and adaptation
• A Divide-and-Conquer Bayesian Approach to Large-Scale Kriging
• A General Framework for Bayes Structured Linear Models
• Bayesian Nonparametric Causal Inference: Information Rates and Learning Algorithms
• Bayes factor consistency
• Semiparametric Bayesian causal inference
• Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators
• Rate-optimal posterior contraction for sparse PCA
• Bayesian Detection of Image Boundaries
• Doubly Decomposing Nonparametric Tensor Regression
• Borrowing strengh in hierarchical Bayes: Posterior concentration of the Dirichlet base measure
• Nonparametric Bayesian inference for reversible multi-dimensional diffusions
• Probabilistic forecast of nonlinear dynamical systems with uncertainty quantification
• Bayesian test of normality versus a Dirichlet process mixture alternative
• Adaptive Bayesian density regression for high-dimensional data
• Convergence Guarantees for Gaussian Process Means With Misspecified Likelihoods and Smoothness
• Optimality of Poisson processes intensity learning with Gaussian processes
• Bayesian linear inverse problems in regularity scales
• On Random Subsampling of Gaussian Process Regression: A Graphon-Based Analysis
• Dimension adaptability of Gaussian process models with variable selection and projection
• Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior
• Comparing and weighting imperfect models using D-probabilities
• Convergence Rates of Variational Posterior Distributions
• On polynomial-time computation of high-dimensional posterior measures by Langevin-type algorithms
• Additive Gaussian Process Regression
• A Fast and Calibrated Computer Model Emulator: An Empirical Bayes Approach
• A Bayesian nonparametric approach to log-concave density estimation
• Adaptive Gaussian Predictive Process Approximation
• Adaptive nonparametric drift estimation for diffusion processes using Faber-Schauder expansions
• Kalman filtering and smoothing for linear wave equations with model error
• Posterior Consistency of Semi-Supervised Regression on Graphs
• On Thompson Sampling with Langevin Algorithms
• Accuracy of Gaussian approximation in nonparametric Bernstein -- von Mises Theorem
• General framework for projection structures
• Modeling Dynamic Functional Connectivity with Latent Factor Gaussian Processes
• Gradient Flow from a Random Walk in Hilbert Space
• Fast learning rate of deep learning via a kernel perspective
• Heavy-Tailed Density Estimation
• Semiparametric Bernstein-von Mises Theorem: Second Order Studies
• Unlabeled Data Help in Graph-Based Semi-Supervised Learning: A Bayesian Nonparametrics Perspective
• Graph Based Gaussian Processes on Restricted Domains
• Designing truncated priors for direct and inverse Bayesian problems
• Nonparametric Bayesian Regression on Manifolds via Brownian Motion
• Posterior contraction rate for non-parametric Bayesian estimation of the dispersion coefficient of a stochastic differential equation
• Consistent non-parametric Bayesian estimation for a time-inhomogeneous Brownian motion
• Invariances of random fields paths, with applications in Gaussian Process Regression
• Flexible Bayesian Dynamic Modeling of Correlation and Covariance Matrices
• Finite Element Representations of Gaussian Processes: Balancing Numerical and Statistical Accuracy
• Oracle posterior contraction rates under hierarchical priors
• Posterior convergence rates in non-linear latent variable models
• A note on Bayesian convergence rates under local prior support conditions
• Bayesian Trend Filtering
• Kriging prediction with isotropic Matérn correlations: Robustness and experimental design
• Nonparametric Bayesian Aggregation for Massive Data
• Locally Adaptive Bayes Nonparametric Regression via Nested Gaussian Processes
• Extrinsic Gaussian processes for regression and classification on manifolds
• On the cost of Bayesian posterior mean strategy for log-concave models
• Learning Temporal Evolution of Spatial Dependence with Generalized Spatiotemporal Gaussian Process Models
• Minimax Error of Interpolation and Optimal Design of Experiments for Variable Fidelity Data
• Semiparametric inference using fractional posteriors
• $L_2$-small ball asymptotics for Gaussian random functions: a survey
• Bayesian factorizations of big sparse tensors
• A note on non-parametric Bayesian estimation for Poisson point processes
• Nonparametric Bayesian methods for one-dimensional diffusion models
• Adaptive Bayesian procedures using random series priors
• A Kernel Approach to Tractable Bayesian Nonparametrics
• Spatiotemporal Besov Priors for Bayesian Inverse Problems
• Exponential-wrapped distributions on symmetric spaces
• Exact L_2-Small Deviation Asymptotics for Some Brownian Functionals
• Posterior contraction rates for constrained deep Gaussian processes in density estimation and classication
• Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals
• Gaussian Process Regression and Classification under Mathematical Constraints with Learning Guarantees
• Posterior Contraction Rates for Gaussian Cox Processes with Non-identically Distributed Data
• Bayesian Probabilistic Numerical Integration with Tree-Based Models
• Prediction based on the Kennedy-O'Hagan calibration model: asymptotic consistency and other properties
• A general approach to small deviation via concentration of measures
• Gaussian Approximation of General Nonparametric Posterior Distributions
• Small deviations for a family of smooth Gaussian processes
• Posterior contraction for deep Gaussian process priors
• On the Statistical Consistency of Risk-Sensitive Bayesian Decision-Making
• A new approach to posterior contraction rates via Wasserstein dynamics
• Consistency and efficiency of Bayesian estimators in generalised linear inverse problems
• On the Convergence of Bayesian Regression Models
• Bayesian estimation of nonlinear Hawkes process
• PAC-Bayesian Bounds for Deep Gaussian Processes
• Semi-parametric Bernstein-von Mises in Linear Inverse Problems
• Thomas Bayes' walk on manifolds
• Semiparametric posterior limits
• BOW: Bayesian Optimization over Windows for Motion Planning in Complex Environments
• Posterior Convergence of Gaussian and General Stochastic Process Regression Under Possible Misspecifications
• A Theoretical Framework for Bayesian Nonparametric Regression
• Interpolation error of misspecified Gaussian process regression
• Quasi-Bayesian Dual Instrumental Variable Regression
• Statistical Guarantees for Transformation Based Models with Applications to Implicit Variational Inference
• Semi-parametric Bayesian Partially Identified Models based on Support Function
• Convergence of Nonparametric Functional Regression Estimates with Functional Responses
• Bayesian Projected Calibration of Computer Models