Explaining Time Series Predictions with Dynamic Masks
arXiv:2106.05303
Abstract
How can we explain the predictions of a machine learning model? When the data is structured as a multivariate time series, this question induces additional difficulties such as the necessity for the explanation to embody the time dependency and the large number of inputs. To address these challenges, we propose dynamic masks (Dynamask). This method produces instance-wise importance scores for each feature at each time step by fitting a perturbation mask to the input sequence. In order to incorporate the time dependency of the data, Dynamask studies the effects of dynamic perturbation operators. In order to tackle the large number of inputs, we propose a scheme to make the feature selection parsimonious (to select no more feature than necessary) and legible (a notion that we detail by making a parallel with information theory). With synthetic and real-world data, we demonstrate that the dynamic underpinning of Dynamask, together with its parsimony, offer a neat improvement in the identification of feature importance over time. The modularity of Dynamask makes it ideal as a plug-in to increase the transparency of a wide range of machine learning models in areas such as medicine and finance, where time series are abundant.
arXiv commentPresented at the Thirty-eighth International Conference on Machine Learning (ICML 2021)
References in corpus (14)
- Axiomatic Attribution for Deep Networks
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
- Learning Important Features Through Propagating Activation Differences
- Visualizing and Understanding Recurrent Networks
- Sanity Checks for Saliency Maps
- Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
- RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
- The (Un)reliability of saliency methods
- TSViz: Demystification of Deep Learning Models for Time-Series Analysis
- Clinical Intervention Prediction and Understanding using Deep Networks
- Interpreting a Recurrent Neural Network's Predictions of ICU Mortality Risk
- Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks
- What went wrong and when? Instance-wise Feature Importance for Time-series Models
- Explanatory Masks for Neural Network Interpretability