Learning stochastic differential equations using RNN with log signature features

Abstract

This paper contributes to the challenge of learning a function on streamed multimodal data through evaluation. The core of the result of our paper is the combination of two quite different approaches to this problem. One comes from the mathematically principled technology of signatures and log-signatures as representations for streamed data, while the other draws on the techniques of recurrent neural networks (RNN). The ability of the former to manage high sample rate streams and the latter to manage large scale nonlinear interactions allows hybrid algorithms that are easy to code, quicker to train, and of lower complexity for a given accuracy. We illustrate the approach by approximating the unknown functional as a controlled differential equation. Linear functionals on solutions of controlled differential equations are the natural universal class of functions on data streams. Following this approach, we propose a hybrid Logsig-RNN algorithm that learns functionals on streamed data. By testing on various datasets, i.e. synthetic data, NTU RGB+D 120 skeletal action data, and Chalearn2013 gesture data, our algorithm achieves the outstanding accuracy with superior efficiency and robustness.

References in corpus (7)

• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
• Sparse arrays of signatures for online character recognition
• Learning from the past, predicting the statistics for the future, learning an evolving system
• Extracting information from the signature of a financial data stream
• The iisignature library: efficient calculation of iterated-integral signatures and log signatures
• A multi-dimensional stream and its signature representation

Cited by in corpus (12)

• Neural Controlled Differential Equations for Irregular Time Series
• Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
• Neural Rough Differential Equations for Long Time Series
• A Generalised Signature Method for Multivariate Time Series Feature Extraction
• Tropical time series, iterated-sums signatures and quasisymmetric functions
• Framing RNN as a kernel method: A neural ODE approach
• Generalised Interpretable Shapelets for Irregular Time Series
• Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory
• Signature features with the visibility transformation
• Path Imputation Strategies for Signature Models of Irregular Time Series
• Robust Hedging GANs
• Deep Signature FBSDE Algorithm