Generalized Spectral Kernels

Abstract

In this paper we propose a family of tractable kernels that is dense in the family of bounded positive semi-definite functions (i.e. can approximate any bounded kernel with arbitrary precision). We start by discussing the case of stationary kernels, and propose a family of spectral kernels that extends existing approaches such as spectral mixture kernels and sparse spectrum kernels. Our extension has two primary advantages. Firstly, unlike existing spectral approaches that yield infinite differentiability, the kernels we introduce allow learning the degree of differentiability of the latent function in Gaussian process (GP) models and functions in the reproducing kernel Hilbert space (RKHS) in other kernel methods. Secondly, we show that some of the kernels we propose require fewer parameters than existing spectral kernels for the same accuracy, thereby leading to faster and more robust inference. Finally, we generalize our approach and propose a flexible and tractable family of spectral kernels that we prove can approximate any continuous bounded nonstationary kernel.

References in corpus (1)

• Structure Discovery in Nonparametric Regression through Compositional Kernel Search

Cited by in corpus (16)

• Differentiable Compositional Kernel Learning for Gaussian Processes
• Non-Stationary Spectral Kernels
• The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings
• Randomized Numerical Linear Algebra: Foundations & Algorithms
• Automated Spectral Kernel Learning
• Adversarial Robustness Guarantees for Gaussian Processes
• Neural Non-Stationary Spectral Kernel
• Marginalised Gaussian Processes with Nested Sampling
• Structured Monte Carlo Sampling for Nonisotropic Distributions via Determinantal Point Processes
• Sparse Spectrum Warped Input Measures for Nonstationary Kernel Learning
• Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
• Marginalising over Stationary Kernels with Bayesian Quadrature
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition
• Convolutional Spectral Kernel Learning
• TripleSpin - a generic compact paradigm for fast machine learning computations
• Gaussian Processes with Skewed Laplace Spectral Mixture Kernels for Long-term Forecasting