GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series

Abstract

Modeling real-world multidimensional time series can be particularly challenging when these are sporadically observed (i.e., sampling is irregular both in time and across dimensions)-such as in the case of clinical patient data. To address these challenges, we propose (1) a continuous-time version of the Gated Recurrent Unit, building upon the recent Neural Ordinary Differential Equations (Chen et al., 2018), and (2) a Bayesian update network that processes the sporadic observations. We bring these two ideas together in our GRU-ODE-Bayes method. We then demonstrate that the proposed method encodes a continuity prior for the latent process and that it can exactly represent the Fokker-Planck dynamics of complex processes driven by a multidimensional stochastic differential equation. Additionally, empirical evaluation shows that our method outperforms the state of the art on both synthetic data and real-world data with applications in healthcare and climate forecast. What is more, the continuity prior is shown to be well suited for low number of samples settings.

Accepted at NeurIPS 2019, Vancouver, Canada

References in corpus (11)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Continuous Time Bayesian Networks
• Latent ODEs for Irregularly-Sampled Time Series
• Deep Kalman Filters
• AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks
• Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier
• Temporal Convolutional Neural Networks for Diagnosis from Lab Tests
• Early Recognition of Sepsis with Gaussian Process Temporal Convolutional Networks and Dynamic Time Warping
• Sparse Multi-Output Gaussian Processes for Medical Time Series Prediction
• Deep Ensemble Tensor Factorization for Longitudinal Patient Trajectories Classification

Cited by in corpus (33)

• Efficiently Modeling Long Sequences with Structured State Spaces
• Neural Controlled Differential Equations for Irregular Time Series
• Deep learning prediction of patient response time course from early data via neural-pharmacokinetic/pharmacodynamic modeling
• Learning Graph ODE for Continuous-Time Sequential Recommendation
• Spatio-Temporal Graph Neural Point Process for Traffic Congestion Event Prediction
• Learning Long-Term Dependencies in Irregularly-Sampled Time Series
• Learning Differential Equations that are Easy to Solve
• Lipschitz Recurrent Neural Networks
• Evolutionary Preference Learning via Graph Nested GRU ODE for Session-based Recommendation
• Neural Rough Differential Equations for Long Time Series
• Interpretable machine learning for high-dimensional trajectories of aging health
• Learning Neural Event Functions for Ordinary Differential Equations
• Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
• Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Prediction
• Multi-Time Attention Networks for Irregularly Sampled Time Series
• Deep Efficient Continuous Manifold Learning for Time Series Modeling
• Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations
• Synthetic Observational Health Data with GANs: from slow adoption to a boom in medical research and ultimately digital twins?
• CKConv: Continuous Kernel Convolution For Sequential Data
• Modeling Irregular Time Series with Continuous Recurrent Units
• Framing RNN as a kernel method: A neural ODE approach
• Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory
• LMT: Longitudinal Mixing Training, a Framework to Predict Disease Progression from a Single Image
• Climate Modeling with Neural Diffusion Equations
• Learning Continuous-Time Dynamics by Stochastic Differential Networks
• A non-autonomous equation discovery method for time signal classification
• Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representations
• Efficient and Accurate Gradients for Neural SDEs
• Variational Dynamic Mixtures
• Manifold-adaptive dimension estimation revisited
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations
• Learning the Dynamics of Sparsely Observed Interacting Systems
• Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data