A Theoretical Framework for Bayesian Nonparametric Regression

Abstract

We develop a unifying framework for Bayesian nonparametric regression to study the rates of contraction with respect to the integrated $L_2$-distance without assuming the regression function space to be uniformly bounded. The framework is very flexible and can be applied to a wide class of nonparametric prior models. Three non-trivial applications of the proposed framework are provided: The finite random series regression of an $\alpha$-Hölder function, with adaptive rates of contraction up to a logarithmic factor; The un-modified block prior regression of an $\alpha$-Sobolev function, with adaptive-and-exact rates of contraction; The Gaussian spline regression of an $\alpha$-Hölder function, with the near-optimal posterior contraction. These applications serve as generalization or complement of their respective results in the literature. Extensions to the fixed-design regression problem and sparse additive models in high dimensions are discussed as well.

References in corpus (9)

• High-dimensional additive modeling
• Rates of contraction of posterior distributions based on Gaussian process priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Sparsity in multiple kernel learning
• Adaptive Bayesian multivariate density estimation with Dirichlet mixtures
• Convergence rates for posterior distributions and adaptive estimation
• Frequentist coverage and sup-norm convergence rate in Gaussian process regression
• Adaptive Supremum Norm Posterior Contraction: Wavelet Spike-and-Slab and Anisotropic Besov Spaces
• Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior