Latent ODEs for Irregularly-Sampled Time Series

Abstract

Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace the recognition network of the recently-proposed Latent ODE model. Both ODE-RNNs and Latent ODEs can naturally handle arbitrary time gaps between observations, and can explicitly model the probability of observation times using Poisson processes. We show experimentally that these ODE-based models outperform their RNN-based counterparts on irregularly-sampled data.

References in corpus (7)

• Auto-Encoding Variational Bayes
• Sequence to Sequence Learning with Neural Networks
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• DeepMind Control Suite
• Discrete Event, Continuous Time RNNs
• Interpolation-Prediction Networks for Irregularly Sampled Time Series

Cited by in corpus (57)

• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series
• Scalable Gradients for Stochastic Differential Equations
• Neural Jump Stochastic Differential Equations
• STDEN: Towards Physics-Guided Neural Networks for Traffic Flow Prediction
• Hamiltonian Graph Networks with ODE Integrators
• Graph Neural Ordinary Differential Equations
• Spatio-Temporal Graph Neural Point Process for Traffic Congestion Event Prediction
• Evolutionary Preference Learning via Graph Nested GRU ODE for Session-based Recommendation
• Continuous Graph Neural Networks
• Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE
• Neural Rough Differential Equations for Long Time Series
• Causal Navigation by Continuous-time Neural Networks
• Learning Neural Event Functions for Ordinary Differential Equations
• Turbulence forecasting via Neural ODE
• Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
• Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Prediction
• Learning Dynamic Alignment via Meta-filter for Few-shot Learning
• Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations
• Learning Multivariate Hawkes Processes at Scale
• Reduced-order Model for Fluid Flows via Neural Ordinary Differential Equations
• Real-time Classification from Short Event-Camera Streams using Input-filtering Neural ODEs
• Modeling Irregular Time Series with Continuous Recurrent Units
• InfoCNF: An Efficient Conditional Continuous Normalizing Flow with Adaptive Solvers
• Probabilistic Numeric Convolutional Neural Networks
• Longitudinal Self-supervised Learning Using Neural Ordinary Differential Equation
• Neural Spatio-Temporal Point Processes
• Policy Analysis using Synthetic Controls in Continuous-Time
• AIRIS: Artificial Intelligence Enhanced Signal Processing in Reconfigurable Intelligent Surface Communications
• LMT: Longitudinal Mixing Training, a Framework to Predict Disease Progression from a Single Image
• Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics
• An Empirical Study: Extensive Deep Temporal Point Process
• Piecewise-constant Neural ODEs
• Forecasting in multivariate irregularly sampled time series with missing values
• Predicting Human Strategies in Simulated Search and Rescue Task
• Continuous-Depth Neural Models for Dynamic Graph Prediction
• NeuralPDR: Neural Differential Equations as surrogate models for Photodissociation Regions
• Differential equations as models of deep neural networks
• Deep learning and differential equations for modeling changes in individual-level latent dynamics between observation periods
• Second-Order Neural ODE Optimizer
• Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representations
• Learning ODEs via Diffeomorphisms for Fast and Robust Integration
• Long Expressive Memory for Sequence Modeling
• Enhancing Human-Machine Teaming for Medical Prognosis Through Neural Ordinary Differential Equations (NODEs)
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations
• A Temporal Kernel Approach for Deep Learning with Continuous-time Information
• Inference of cell dynamics on perturbation data using adjoint sensitivity
• Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
• Progressive Growing of Neural ODEs
• Robust Neural Networks inspired by Strong Stability Preserving Runge-Kutta methods
• NeurInt : Learning to Interpolate through Neural ODEs
• Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Series
• SurvODE: Extrapolating Gene Expression Distribution for Early Cancer Identification
• RNN Training along Locally Optimal Trajectories via Frank-Wolfe Algorithm
• As easy as APC: overcoming missing data and class imbalance in time series with self-supervised learning
• STR-GODEs: Spatial-Temporal-Ridership Graph ODEs for Metro Ridership Prediction