Hilbert Space Methods for Reduced-Rank Gaussian Process Regression

Abstract

This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of ${\mathbb{R}}^d$. On this approximate eigenbasis the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as $\mathcal{O}(n{m}^2)$ (initial) and $\mathcal{O}(m^3)$ (hyperparameter learning) with $m$ basis functions and $n$ data points. Furthermore, the basis functions are independent of the parameters of the covariance function, which allows for very fast hyperparameter learning. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. We also show that the convergence rate of the truncation error is independent of the input dimensionality provided that the differentiability order of the covariance function is increases appropriately, and for the squared exponential covariance function it is always bounded by $\sim 1/m$ regardless of the input dimensionality. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.

Cited by in corpus (19)

• Physics-informed machine learning for Structural Health Monitoring
• Detecting and diagnosing prior and likelihood sensitivity with power-scaling
• Gaussian Process Manifold Interpolation for Probabilistic Atrial Activation Maps and Uncertain Conduction Velocity
• Constraining Gaussian processes for physics-informed acoustic emission mapping
• Variational Inference at Glacier Scale
• Intragranular Strain Estimation in Far-Field Scanning X-ray Diffraction using a Gaussian Processes
• Low-rank statistical finite elements for scalable model-data synthesis
• Probabilistic approach to limited-data computed tomography reconstruction
• Rao-Blackwellized Particle Smoothing for Simultaneous Localization and Mapping
• Learning battery model parameter dynamics from data with recursive Gaussian process regression
• Estimating fine age structure and time trends in human contact patterns from coarse contact data: the Bayesian rate consistency model
• Gaussian Process-Gated Hierarchical Mixtures of Experts
• The SPDE approach for Gaussian and non-Gaussian fields: 10 years and still running
• Kalman Filter-Based Distributed Gaussian Process for Unknown Scalar Field Estimation in Wireless Sensor Networks
• Learning-Based Optimal Control with Performance Guarantees for Unknown Systems with Latent States
• Computationally efficient multi-level Gaussian process regression for functional data observed under completely or partially regular sampling designs
• A Kernel-Based Approach for Modelling Gaussian Processes with Functional Information
• Large-scale magnetic field maps using structured kernel interpolation for Gaussian process regression
• Hilbert space methods for approximating multi-output latent variable Gaussian processes