TSInsight: A local-global attribution framework for interpretability in time-series data
arXiv:2004.02958
Abstract
With the rise in the employment of deep learning methods in safety-critical scenarios, interpretability is more essential than ever before. Although many different directions regarding interpretability have been explored for visual modalities, time-series data has been neglected with only a handful of methods tested due to their poor intelligibility. We approach the problem of interpretability in a novel way by proposing TSInsight where we attach an auto-encoder to the classifier with a sparsity-inducing norm on its output and fine-tune it based on the gradients from the classifier and a reconstruction penalty. TSInsight learns to preserve features that are important for prediction by the classifier and suppresses those that are irrelevant i.e. serves as a feature attribution method to boost interpretability. In contrast to most other attribution frameworks, TSInsight is capable of generating both instance-based and model-based explanations. We evaluated TSInsight along with 9 other commonly used attribution methods on 8 different time-series datasets to validate its efficacy. Evaluation results show that TSInsight naturally achieves output space contraction, therefore, is an effective tool for the interpretability of deep time-series models.
References in corpus (11)
- Axiomatic Attribution for Deep Networks
- Striving for Simplicity: The All Convolutional Net
- Understanding Neural Networks Through Deep Visualization
- SmoothGrad: removing noise by adding noise
- Sanity Checks for Saliency Maps
- Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
- Visualizing and Understanding Convolutional Networks
- Towards Robust Interpretability with Self-Explaining Neural Networks
- TSViz: Demystification of Deep Learning Models for Time-Series Analysis
- Explaining Deep Learning Models - A Bayesian Non-parametric Approach
- Feature Importance Measure for Non-linear Learning Algorithms