Additive Gaussian Processes

Abstract

We introduce a Gaussian process model of functions which are additive. An additive function is one which decomposes into a sum of low-dimensional functions, each depending on only a subset of the input variables. Additive GPs generalize both Generalized Additive Models, and the standard GP models which use squared-exponential kernels. Hyperparameter learning in this model can be seen as Bayesian Hierarchical Kernel Learning (HKL). We introduce an expressive but tractable parameterization of the kernel function, which allows efficient evaluation of all input interaction terms, whose number is exponential in the input dimension. The additional structure discoverable by this model results in increased interpretability, as well as state-of-the-art predictive power in regression tasks.

Appearing in Neural Information Processing Systems 2011

References in corpus (2)

• Expectation Propagation for approximate Bayesian inference
• High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning

Cited by in corpus (46)

• Structure Discovery in Nonparametric Regression through Compositional Kernel Search
• High Dimensional Bayesian Optimisation and Bandits via Additive Models
• Variational Fourier features for Gaussian processes
• Avoiding pathologies in very deep networks
• A Bayesian Additive Model for Understanding Public Transport Usage in Special Events
• Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
• Differentiable Compositional Kernel Learning for Gaussian Processes
• Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks
• Gaussian Processes for Survival Analysis
• Multiresolution Gaussian Processes
• Generalized Spectral Kernels
• Local Gaussian Processes for Efficient Fine-Grained Traffic Speed Prediction
• Easy representation of multivariate functions with low-dimensional terms via Gaussian process regression kernel design: applications to machine learning of potential energy surfaces and kinetic energy densities from sparse data
• Bayesian optimization for inverse problems in time-dependent quantum dynamics
• Bayesian Semi-supervised Learning with Graph Gaussian Processes
• Gaussian Process States: A data-driven representation of quantum many-body physics
• A Gaussian Process perspective on Convolutional Neural Networks
• Randomly Projected Additive Gaussian Processes for Regression
• lgpr: An interpretable nonparametric method for inferring covariate effects from longitudinal data
• Additive Tree-Structured Conditional Parameter Spaces in Bayesian Optimization: A Novel Covariance Function and a Fast Implementation
• Recovering Latent Signals from a Mixture of Measurements using a Gaussian Process Prior
• Fast increased fidelity approximate Gibbs samplers for Bayesian Gaussian process regression
• Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior
• Bayesian Optimization over Hybrid Spaces
• Correlated Weights in Infinite Limits of Deep Convolutional Neural Networks
• Accelerated scale bridging with sparsely approximated Gaussian learning
• Additive Tree-Structured Covariance Function for Conditional Parameter Spaces in Bayesian Optimization
• Polynomial-Chaos-based Kriging
• No-Regret Bayesian Optimization with Unknown Hyperparameters
• Invariances of random fields paths, with applications in Gaussian Process Regression
• Scaling Multidimensional Inference for Structured Gaussian Processes
• Longitudinal Deep Kernel Gaussian Process Regression
• Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey
• On ANOVA decompositions of kernels and Gaussian random field paths
• Autoregressive Density Modeling with the Gaussian Process Mixture Transition Distribution
• Projection Pursuit Gaussian Process Regression
• Learning Nonparametric Volterra Kernels with Gaussian Processes
• Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior
• String Gaussian Process Kernels
• Multiple Kernel Learning and Automatic Subspace Relevance Determination for High-dimensional Neuroimaging Data
• String and Membrane Gaussian Processes
• Gaussian Processes for Individualized Continuous Treatment Rule Estimation
• Large-Scale Learning with Fourier Features and Tensor Decompositions
• Active learning for enumerating local minima based on Gaussian process derivatives
• Kernel Identification Through Transformers
• Interpolation and extrapolation of global potential energy surfaces for polyatomic systems by Gaussian processes with composite kernels