Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks

Abstract

Deep neural networks have shown promising results for various clinical prediction tasks such as diagnosis, mortality prediction, predicting duration of stay in hospital, etc. However, training deep networks -- such as those based on Recurrent Neural Networks (RNNs) -- requires large labeled data, high computational resources, and significant hyperparameter tuning effort. In this work, we investigate as to what extent can transfer learning address these issues when using deep RNNs to model multivariate clinical time series. We consider transferring the knowledge captured in an RNN trained on several source tasks simultaneously using a large labeled dataset to build the model for a target task with limited labeled data. An RNN pre-trained on several tasks provides generic features, which are then used to build simpler linear models for new target tasks without training task-specific RNNs. For evaluation, we train a deep RNN to identify several patient phenotypes on time series from MIMIC-III database, and then use the features extracted using that RNN to build classifiers for identifying previously unseen phenotypes, and also for a seemingly unrelated task of in-hospital mortality. We demonstrate that (i) models trained on features extracted using pre-trained RNN outperform or, in the worst case, perform as well as task-specific RNNs; (ii) the models using features from pre-trained models are more robust to the size of labeled data than task-specific RNNs; and (iii) features extracted using pre-trained RNN are generic enough and perform better than typical statistical hand-crafted features.

arXiv commentAccepted at Machine Learning for Medicine and Healthcare Workshop at ACM KDD 2018 Conference

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Neural Machine Translation by Jointly Learning to Align and Translate
• On the difficulty of training Recurrent Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Learning to Diagnose with LSTM Recurrent Neural Networks
• Recurrent Neural Networks for Multivariate Time Series with Missing Values
• TimeNet: Pre-trained deep recurrent neural network for time series classification
• Towards a universal neural network encoder for time series
• Transfer Learning for Named-Entity Recognition with Neural Networks
• Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets

Cited by in corpus (4)

• AdaRNN: Adaptive Learning and Forecasting of Time Series
• Evaluating Progress on Machine Learning for Longitudinal Electronic Healthcare Data
• CovidCare: Transferring Knowledge from Existing EMR to Emerging Epidemic for Interpretable Prognosis
• Neuroevolutionary Transfer Learning of Deep Recurrent Neural Networks through Network-Aware Adaptation