Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodes
arXiv:2004.12538
Abstract
Recently deep neural networks demonstrate competitive performances in classification and regression tasks for many temporal or sequential data. However, it is still hard to understand the classification mechanisms of temporal deep neural networks. In this paper, we propose two new frameworks to visualize temporal representations learned from deep neural networks. Given input data and output, our algorithm interprets the decision of temporal neural network by extracting highly activated periods and visualizes a sub-sequence of input data which contributes to activate the units. Furthermore, we characterize such sub-sequences with clustering and calculate the uncertainty of the suggested type and actual data. We also suggest Layer-wise Relevance from the output of a unit, not from the final output, with backward Monte-Carlo dropout to show the relevance scores of each input point to activate units with providing a visual representation of the uncertainty about this impact.
References in corpus (6)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- WaveNet: A Generative Model for Raw Audio
- Striving for Simplicity: The All Convolutional Net
- GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
- From Clustering to Cluster Explanations via Neural Networks
- An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks