Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances

Abstract

We develop a Bayesian approach to learning from sequential data by using Gaussian processes (GPs) with so-called signature kernels as covariance functions. This allows to make sequences of different length comparable and to rely on strong theoretical results from stochastic analysis. Signatures capture sequential structure with tensors that can scale unfavourably in sequence length and state space dimension. To deal with this, we introduce a sparse variational approach with inducing tensors. We then combine the resulting GP with LSTMs and GRUs to build larger models that leverage the strengths of each of these approaches and benchmark the resulting GPs on multivariate time series (TS) classification datasets. Code available at https://github.com/tgcsaba/GPSig.

arXiv commentNear camera ready version for ICML 2020. Previous title: "Variational Gaussian Processes with Signature Covariances"

Cited by in corpus (9)

• Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
• A Generalised Signature Method for Multivariate Time Series Feature Extraction
• Framing RNN as a kernel method: A neural ODE approach
• Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory
• Signatured Deep Fictitious Play for Mean Field Games with Common Noise
• Seq2Tens: An Efficient Representation of Sequences by Low-Rank Tensor Projections
• SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data
• Path Imputation Strategies for Signature Models of Irregular Time Series
• Efficient and Accurate Gradients for Neural SDEs