Switched linear projections for neural network interpretability
arXiv:1909.11275
Abstract
We introduce switched linear projections for expressing the activity of a neuron in a deep neural network in terms of a single linear projection in the input space. The method works by isolating the active subnetwork, a series of linear transformations, that determine the entire computation of the network for a given input instance. With these projections we can decompose activity in any hidden layer into patterns detected in a given input instance. We also propose that in ReLU networks it is instructive and meaningful to examine patterns that deactivate the neurons in a hidden layer, something that is implicitly ignored by the existing interpretability methods tracking solely the active aspect of the network's computation.
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Axiomatic Attribution for Deep Networks
- Learning Important Features Through Propagating Activation Differences
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
- Sanity Checks for Saliency Maps
- A Closer Look at Memorization in Deep Networks
- Deep ReLU Networks Have Surprisingly Few Activation Patterns