Sparse Spectrum Warped Input Measures for Nonstationary Kernel Learning

Abstract

We establish a general form of explicit, input-dependent, measure-valued warpings for learning nonstationary kernels. While stationary kernels are ubiquitous and simple to use, they struggle to adapt to functions that vary in smoothness with respect to the input. The proposed learning algorithm warps inputs as conditional Gaussian measures that control the smoothness of a standard stationary kernel. This construction allows us to capture non-stationary patterns in the data and provides intuitive inductive bias. The resulting method is based on sparse spectrum Gaussian processes, enabling closed-form solutions, and is extensible to a stacked construction to capture more complex patterns. The method is extensively validated alongside related algorithms on synthetic and real world datasets. We demonstrate a remarkable efficiency in the number of parameters of the warping functions in learning problems with both small and large data regimes.

arXiv commentAccepted version for NeurIPS 2020

References in corpus (14)

• Scalable Variational Gaussian Process Classification
• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• Gaussian Process Regression Networks
• Input Warping for Bayesian Optimization of Non-stationary Functions
• Random Feature Expansions for Deep Gaussian Processes
• Differentiable Compositional Kernel Learning for Gaussian Processes
• Non-Stationary Spectral Kernels
• Bayesian Optimization with Adaptive Kernels for Robot Control
• Gaussian Process Regression with Heteroscedastic or Non-Gaussian Residuals
• Linearly constrained Gaussian processes
• Gaussian Process Conditional Density Estimation
• Generalized Spectral Kernels
• Deep learning with differential Gaussian process flows
• Bayesian optimisation under uncertain inputs