Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodes
arXiv:2004.12538
Abstract
Recently deep neural networks demonstrate competitive performances in classification and regression tasks for many temporal or sequential data. However, it is still hard to understand the classification mechanisms of temporal deep neural networks. In this paper, we propose two new frameworks to visualize temporal representations learned from deep neural networks. Given input data and output, our algorithm interprets the decision of temporal neural network by extracting highly activated periods and visualizes a sub-sequence of input data which contributes to activate the units. Furthermore, we characterize such sub-sequences with clustering and calculate the uncertainty of the suggested type and actual data. We also suggest Layer-wise Relevance from the output of a unit, not from the final output, with backward Monte-Carlo dropout to show the relevance scores of each input point to activate units with providing a visual representation of the uncertainty about this impact.
References in corpus (12)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- LSTM: A Search Space Odyssey
- WaveNet: A Generative Model for Raw Audio
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- Striving for Simplicity: The All Convolutional Net
- Learning Important Features Through Propagating Activation Differences
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- Learning Deep Features for Discriminative Localization
- GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
- From Clustering to Cluster Explanations via Neural Networks
- Generating Visual Explanations
- An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks