Neural Controlled Differential Equations for Online Prediction Tasks
arXiv:2106.11028
Abstract
Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in \textit{online prediction tasks}, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that interpolation schemes for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks.
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
- Neural Ordinary Differential Equations
- Neural Jump Stochastic Differential Equations
- The UEA multivariate time series classification archive, 2018
- Neural Rough Differential Equations for Long Time Series
- Learning Neural Event Functions for Ordinary Differential Equations
- Monash University, UEA, UCR Time Series Extrinsic Regression Archive
- A Generalised Signature Method for Multivariate Time Series Feature Extraction
- Policy Analysis using Synthetic Controls in Continuous-Time