Recurrent Neural Networks for Multivariate Time Series with Missing Values

Abstract

Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provides useful insights for better understanding and utilization of missing values in time series analysis.

References in corpus (7)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• Learning to Diagnose with LSTM Recurrent Neural Networks
• Doctor AI: Predicting Clinical Events via Recurrent Neural Networks
• Modeling Missing Data in Clinical Time Series with RNNs
• DeepCare: A Deep Dynamic Memory Model for Predictive Medicine

Cited by in corpus (30)

• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record
• GRAM: Graph-based Attention Model for Healthcare Representation Learning
• ISeeU: Visually interpretable deep learning for mortality prediction inside the ICU
• Clinical Intervention Prediction and Understanding using Deep Networks
• Neural Document Embeddings for Intensive Care Patient Mortality Prediction
• Dropout Feature Ranking for Deep Learning Models
• Improving Factor-Based Quantitative Investing by Forecasting Company Fundamentals
• Deepr: A Convolutional Net for Medical Records
• Machine Learning in High Energy Physics: A review of heavy-flavor jet tagging at the LHC
• Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
• RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data
• Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks
• Not to Cry Wolf: Distantly Supervised Multitask Learning in Critical Care
• Deep Learning to Attend to Risk in ICU
• Learning from Clinical Judgments: Semi-Markov-Modulated Marked Hawkes Processes for Risk Prognosis
• Evolutionary Neural AutoML for Deep Learning
• Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning
• Learning Hierarchical Representations of Electronic Health Records for Clinical Outcome Prediction
• Long-Term Mobile Traffic Forecasting Using Deep Spatio-Temporal Neural Networks
• A novel stacked hybrid autoencoder for imputing LISA data gaps
• Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification
• Autoregressive-Model-Based Methods for Online Time Series Prediction with Missing Values: an Experimental Evaluation
• Explainable Failure Predictions with RNN Classifiers based on Time Series Data
• Learning compressed representations of blood samples time series with missing data
• An Unsupervised Multivariate Time Series Kernel Approach for Identifying Patients with Surgical Site Infection from Blood Samples
• Sharing pattern submodels for prediction with missing values
• Precipitation Nowcasting: Leveraging bidirectional LSTM and 1D CNN
• Long-term Blood Pressure Prediction with Deep Recurrent Neural Networks
• Medi-Care AI: Predicting Medications From Billing Codes via Robust Recurrent Neural Networks