Additive Gaussian Process Regression

Abstract

Additive-interactive regression has recently been shown to offer attractive minimax error rates over traditional nonparametric multivariate regression in a wide variety of settings, including cases where the predictor count is much larger than the sample size and many of the predictors have important effects on the response, potentially through complex interactions. We present a Bayesian implementation of additive-interactive regression using an additive Gaussian process (AGP) prior and develop an efficient Markov chain sampler that extends stochastic search variable selection in this setting. Careful prior and hyper-parameter specification are developed in light of performance and computational considerations, and key innovations address difficulties in exploring a joint posterior distribution over multiple subsets of high dimensional predictor inclusion vectors. The method offers state-of-the-art support and interaction recovery while improving dramatically over competitors in terms of prediction accuracy on a diverse set of simulated and real data. Results from real data studies provide strong evidence that the additive-interactive framework is an attractive modeling platform for high-dimensional nonparametric regression.

28 pages; 9 figures; 5 tables

References in corpus (6)

• Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification
• Rates of contraction of posterior distributions based on Gaussian process priors
• Anisotropic function estimation using multi-bandwidth Gaussian processes
• Selection of variables and dimension reduction in high-dimensional non-parametric regression
• Dimension adaptability of Gaussian process models with variable selection and projection
• Adaptive Gaussian Predictive Process Approximation

Cited by in corpus (2)

• Randomly Projected Additive Gaussian Processes for Regression
• Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes