Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo

Abstract

We present a novel approach for fully non-stationary Gaussian process regression (GPR), where all three key parameters -- noise variance, signal variance and lengthscale -- can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. We propose to infer full parameter posterior with Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with model gradients. We also learn the MAP solution from the posterior by gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the nonstationary GPR is shown to be necessary for modeling realistic input-dependent dynamics, while it performs comparably to conventional stationary or previous non-stationary GPR models otherwise.

Cited by in corpus (15)

• Non-Stationary Spectral Kernels
• Deep State-Space Gaussian Processes
• Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes
• Deep Gaussian Covariance Network
• Sequential Gaussian Processes for Online Learning of Nonstationary Functions
• Large-scale Heteroscedastic Regression via Gaussian Process
• How Deep Are Deep Gaussian Processes?
• State-space deep Gaussian processes with applications
• Scalable Gaussian Process Regression for Kernels with a Non-Stationary Phase
• Bayesian Functional Mixed-Effects Model with Gaussian Process Responses for Wavelet Spectra of Spatiotemporal Colonic Manometry Signals
• Nonstationary Multivariate Gaussian Processes for Electronic Health Records
• Inference of Dynamic Regimes in the Microbiome
• Quantifying experimental edge plasma evolution via multidimensional adaptive Gaussian process regression
• Hierarchical Non-Stationary Temporal Gaussian Processes With $L^1$-Regularization
• Learning Arbitrary Quantities of Interest from Expensive Black-Box Functions through Bayesian Sequential Optimal Design