Visual Summary of Value-level Feature Attribution in Prediction Classes with Recurrent Neural Networks
arXiv:2001.08379
Abstract
Deep Recurrent Neural Networks (RNN) is increasingly used in decision-making with temporal sequences. However, understanding how RNN models produce final predictions remains a major challenge. Existing work on interpreting RNN models for sequence predictions often focuses on explaining predictions for individual data instances (e.g., patients or students). Because state-of-the-art predictive models are formed with millions of parameters optimized over millions of instances, explaining predictions for single data instances can easily miss a bigger picture. Besides, many outperforming RNN models use multi-hot encoding to represent the presence/absence of features, where the interpretability of feature value attribution is missing. We present ViSFA, an interactive system that visually summarizes feature attribution over time for different feature values. ViSFA scales to large data such as the MIMIC dataset containing the electronic health records of 1.2 million high-dimensional temporal events. We demonstrate that ViSFA can help us reason RNN prediction and uncover insights from data by distilling complex attribution into compact and easy-to-interpret visualizations.
arXiv comment10 pages
References in corpus (10)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Towards A Rigorous Science of Interpretable Machine Learning
- Teaching Machines to Read and Comprehend
- Learning Important Features Through Propagating Activation Differences
- Recent Trends in Deep Learning Based Natural Language Processing
- Visualizing and Understanding Recurrent Networks
- Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
- RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
- Do Convolutional Neural Networks Learn Class Hierarchy?
- Predicting purchasing intent: Automatic Feature Learning using Recurrent Neural Networks