Standing Wave Decomposition Gaussian Process

Abstract

We propose a Standing Wave Decomposition (SWD) approximation to Gaussian Process regression (GP). GP involves a costly matrix inversion operation, which limits applicability to large data analysis. For an input space that can be approximated by a grid and when correlations among data are short-ranged, the kernel matrix inversion can be replaced by analytic diagonalization using the SWD. We show that this approach applies to uni- and multi-dimensional input data, extends to include longer-range correlations, and the grid can be in a latent space and used as inducing points. Through simulations, we show that our approximate method applied to the squared exponential kernel outperforms existing methods in predictive accuracy per unit time in the regime where data are plentiful. Our SWD-GP is recommended for regression analyses where there is a relatively large amount of data and/or there are constraints on computation time.

arXiv comment10 pages, 8 figures; updated version includes a modified introduction and a new discussion on time complexity of our approximated GP method. New references are added. Simulation package will be announced later; updated with discussion of validity of perturbation treatment of Eq. (25) with added Fig. 6 as evidence; simulation code at https://github.com/CoDaS-Lab/LG-SWD-GP

References in corpus (11)

• Quantum algorithm for solving linear systems of equations
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• Machine learning with systematic density-functional theory calculations: Application to melting temperatures of single and binary component solids
• Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (2013)
• Generalized arcsine laws for fractional Brownian motion
• Statistical Gravitational Waveform Models: What to Simulate Next?
• Understanding Probabilistic Sparse Gaussian Process Approximations
• Remote Sensing Image Classification with Large Scale Gaussian Processes
• Continuous-variable quantum Gaussian process regression and quantum singular value decomposition of non-sparse low rank matrices
• Inferring the spatiotemporal DNA replication program from noisy data
• Blitzkriging: Kronecker-structured Stochastic Gaussian Processes

Cited by in corpus (2)

• Estimating regression errors without ground truth values
• Bayesian inference with finitely wide neural networks