Path Imputation Strategies for Signature Models of Irregular Time Series
arXiv:2005.12359
Abstract
The signature transform is a 'universal nonlinearity' on the space of continuous vector-valued paths, and has received attention for use in machine learning on time series. However, real-world temporal data is typically observed at discrete points in time, and must first be transformed into a continuous path before signature techniques can be applied. We make this step explicit by characterising it as an imputation problem, and empirically assess the impact of various imputation strategies when applying signature-based neural nets to irregular time series data. For one of these strategies, Gaussian process (GP) adapters, we propose an extension~(GP-PoM) that makes uncertainty information directly available to the subsequent classifier while at the same time preventing costly Monte-Carlo (MC) sampling. In our experiments, we find that the choice of imputation drastically affects shallow signature models, whereas deeper architectures are more robust. Next, we observe that uncertainty-aware predictions (based on GP-PoM or indicator imputations) are beneficial for predictive performance, even compared to the uncertainty-aware training of conventional GP adapters. In conclusion, we have demonstrated that the path construction is indeed crucial for signature models and that our proposed strategy leads to competitive performance in general, while improving robustness of signature models in particular.
References in corpus (18)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
- Rough paths, Signatures and the modelling of functions on streams
- Learning from the past, predicting the statistics for the future, learning an evolving system
- Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
- Deep Signature Transforms
- A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification
- Early Recognition of Sepsis with Gaussian Process Temporal Convolutional Networks and Dynamic Time Warping
- Signature moments to characterize laws of stochastic processes
- Learning stochastic differential equations using RNN with log signature features
- Interpolation-Prediction Networks for Irregularly Sampled Time Series
- Set Functions for Time Series
- The iisignature library: efficient calculation of iterated-integral signatures and log signatures
- Application of the Signature Method to Pattern Recognition in the CEQUEL Clinical Trial
- Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances
- Embedding and learning with signatures