RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism

Abstract

Accuracy and interpretability are two dominant features of successful predictive models. Typically, a choice must be made in favor of complex black box models such as recurrent neural networks (RNN) for accuracy versus less accurate but more interpretable traditional models such as logistic regression. This tradeoff poses challenges in medicine where both accuracy and interpretability are important. We addressed this challenge by developing the REverse Time AttentIoN model (RETAIN) for application to Electronic Health Records (EHR) data. RETAIN achieves high accuracy while remaining clinically interpretable and is based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits (e.g. key diagnoses). RETAIN mimics physician practice by attending the EHR data in a reverse time order so that recent clinical visits are likely to receive higher attention. RETAIN was tested on a large health system EHR dataset with 14 million visits completed by 263K patients over an 8 year period and demonstrated predictive accuracy and computational scalability comparable to state-of-the-art methods such as RNN, and ease of interpretability comparable to traditional models.

Accepted at Neural Information Processing Systems (NIPS) 2016

References in corpus (8)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• ADADELTA: An Adaptive Learning Rate Method
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• DRAW: A Recurrent Neural Network For Image Generation
• Visualizing and Understanding Recurrent Networks
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• Doctor AI: Predicting Clinical Events via Recurrent Neural Networks
• Distilling Knowledge from Deep Networks with Applications to Healthcare Domain

Cited by in corpus (16)

• Multitask learning and benchmarking with clinical time series data
• TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
• BioConceptVec: creating and evaluating literature-based biomedical concept embeddings on a large scale
• MGP-AttTCN: An Interpretable Machine Learning Model for the Prediction of Sepsis
• Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
• Synthesize High-dimensional Longitudinal Electronic Health Records via Hierarchical Autoregressive Language Model
• Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks
• All Data Inclusive, Deep Learning Models to Predict Critical Events in the Medical Information Mart for Intensive Care III Database (MIMIC III)
• Explaining Neural Networks Semantically and Quantitatively
• Explainable Health Risk Predictor with Transformer-based Medicare Claim Encoder
• Learning Predictive and Interpretable Timeseries Summaries from ICU Data
• Predicting Opioid Use Disorder from Longitudinal Healthcare Data using Multi-stream Transformer
• Efficient Symptom Inquiring and Diagnosis via Adaptive Alignment of Reinforcement Learning and Classification
• A Simple and Interpretable Predictive Model for Healthcare
• HealthAdvisor: Recommendation System for Wearable Technologies enabling Proactive Health Monitoring
• Temporal Clustering with External Memory Network for Disease Progression Modeling