Interpreting a Recurrent Neural Network's Predictions of ICU Mortality Risk

Abstract

Deep learning has demonstrated success in many applications; however, their use in healthcare has been limited due to the lack of transparency into how they generate predictions. Algorithms such as Recurrent Neural Networks (RNNs) when applied to Electronic Medical Records (EMR) introduce additional barriers to transparency because of the sequential processing of the RNN and the multi-modal nature of EMR data. This work seeks to improve transparency by: 1) introducing Learned Binary Masks (LBM) as a method for identifying which EMR variables contributed to an RNN model's risk of mortality (ROM) predictions for critically ill children; and 2) applying KernelSHAP for the same purpose. Given an individual patient, LBM and KernelSHAP both generate an attribution matrix that shows the contribution of each input feature to the RNN's sequence of predictions for that patient. Attribution matrices can be aggregated in many ways to facilitate different levels of analysis of the RNN model and its predictions. Presented are three methods of aggregations and analyses: 1) over volatile time periods within individual patient predictions, 2) over populations of ICU patients sharing specific diagnoses, and 3) across the general population of critically ill children.

References in corpus (15)

• Towards A Rigorous Science of Interpretable Machine Learning
• Scalable and accurate deep learning for electronic health records
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Visualizing and Understanding Recurrent Networks
• Consistent Individualized Feature Attribution for Tree Ensembles
• Detecting Cancer Metastases on Gigapixel Pathology Images
• Learning to Diagnose with LSTM Recurrent Neural Networks
• Explainable AI for Trees: From Local Explanations to Global Understanding
• Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record
• Visualizing and Understanding Neural Models in NLP
• The (Un)reliability of saliency methods
• Clinical Intervention Prediction and Understanding using Deep Networks
• The Dependence of Machine Learning on Electronic Medical Record Quality
• A Large-scale Multimodal Study for Predicting Mortality Risk Using Minimal and Low Parameter Models and Separable Risk Assessment
• The Impact of Extraneous Variables on the Performance of Recurrent Neural Network Models in Clinical Tasks

Cited by in corpus (3)

• TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
• Explaining Time Series Predictions with Dynamic Masks
• Discovering long term dependencies in noisy time series data using deep learning