Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition
arXiv:2106.05992
Abstract
We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We propose the harmonic kernel decomposition (HKD), which uses Fourier series to decompose a kernel as a sum of orthogonal kernels. Our variational approximation exploits this orthogonality to enable a large number of inducing points at a low computational cost. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections, and it significantly outperforms standard variational methods in scalability and accuracy. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
ICML2021, 21 pages
References in corpus (14)
- Gaussian Processes for Big Data
- Scalable Variational Gaussian Process Classification
- Gaussian Process Kernels for Pattern Discovery and Extrapolation
- Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
- Orthogonal Random Features
- Variational Fourier features for Gaussian processes
- Thoughts on Massively Scalable Gaussian Processes
- Matérn Gaussian processes on Riemannian manifolds
- Generalized Spectral Kernels
- Orthogonally Decoupled Variational Gaussian Processes
- Sparse Gaussian Processes with Spherical Harmonic Features
- Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)
- Function-Space Distributions over Kernels
- Variational Orthogonal Features