Breaking Batch Normalization for better explainability of Deep Neural Networks through Layer-wise Relevance Propagation

Abstract

The lack of transparency of neural networks stays a major break for their use. The Layerwise Relevance Propagation technique builds heat-maps representing the relevance of each input in the model s decision. The relevance spreads backward from the last to the first layer of the Deep Neural Network. Layer-wise Relevance Propagation does not manage normalization layers, in this work we suggest a method to include normalization layers. Specifically, we build an equivalent network fusing normalization layers and convolutional or fully connected layers. Heatmaps obtained with our method on MNIST and CIFAR 10 datasets are more accurate for convolutional layers. Our study also prevents from using Layerwise Relevance Propagation with networks including a combination of connected layers and normalization layer.

References in corpus (11)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Deep Residual Learning for Image Recognition
• Methods for Interpreting and Understanding Deep Neural Networks
• Learning Important Features Through Propagating Activation Differences
• Densely Connected Convolutional Networks
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Rethinking the Inception Architecture for Computer Vision
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases

Cited by in corpus (2)

• Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning
• Unsupervised Model Drift Estimation with Batch Normalization Statistics for Dataset Shift Detection and Model Selection