SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data

Abstract

Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series.

arXiv commentPublished at ICML 2021

References in corpus (8)

• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Variational Fourier features for Gaussian processes
• Rough paths, Signatures and the modelling of functions on streams
• Using path signatures to predict a diagnosis of Alzheimer's disease
• The iisignature library: efficient calculation of iterated-integral signatures and log signatures
• Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances
• Sparse Gaussian Processes with Spherical Harmonic Features
• Variational Orthogonal Features