Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
arXiv:2004.12524
Abstract
Deep learning continues to revolutionize an ever-growing number of critical application areas including healthcare, transportation, finance, and basic sciences. Despite their increased predictive power, model transparency and human explainability remain a significant challenge due to the "black box" nature of modern deep learning models. In many cases the desired balance between interpretability and performance is predominately task specific. Human-centric domains such as healthcare necessitate a renewed focus on understanding how and why these frameworks are arriving at critical and potentially life-or-death decisions. Given the quantity of research and empirical successes of deep learning for computer vision, most of the existing interpretability research has focused on image processing techniques. Comparatively, less attention has been paid to interpreting deep learning frameworks using sequential data. Given recent deep learning advancements in highly sequential domains such as natural language processing and physiological signal processing, the need for deep sequential explanations is at an all-time high. In this paper, we review current techniques for interpreting deep learning techniques involving sequential data, identify similarities to non-sequential methods, and discuss current limitations and future avenues of sequential interpretability research.
References in corpus (35)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Deep learning with convolutional neural networks for EEG decoding and visualization
- Towards A Rigorous Science of Interpretable Machine Learning
- Learning Important Features Through Propagating Activation Differences
- Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
- A Structured Self-attentive Sentence Embedding
- Recent Trends in Deep Learning Based Natural Language Processing
- Visualizing and Understanding Recurrent Networks
- Reasoning about Entailment with Neural Attention
- Understanding Neural Networks through Representation Erasure
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- "Why Should I Trust You?": Explaining the Predictions of Any Classifier
- RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
- ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
- Long Short-Term Memory-Networks for Machine Reading
- Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record
- Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
- Modeling Coverage for Neural Machine Translation
- Measuring the Effects of Data Parallelism on Neural Network Training
- Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
- Distilling Knowledge from Deep Networks with Applications to Healthcare Domain
- TSViz: Demystification of Deep Learning Models for Time-Series Analysis
- ISeeU: Visually interpretable deep learning for mortality prediction inside the ICU
- Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
- Characterizing Driving Styles with Deep Learning
- Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
- Automatic Rule Extraction from Long Short Term Memory Networks
- SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties
- Deep Motif: Visualizing Genomic Sequence Classifications
- Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
- An interpretable LSTM neural network for autoregressive exogenous model
- Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks
- Interpretable Deep Convolutional Neural Networks via Meta-learning
- Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis