Understanding Individual Decisions of CNNs via Contrastive Backpropagation

Abstract

A number of backpropagation-based approaches such as DeConvNets, vanilla Gradient Visualization and Guided Backpropagation have been proposed to better understand individual decisions of deep convolutional neural networks. The saliency maps produced by them are proven to be non-discriminative. Recently, the Layer-wise Relevance Propagation (LRP) approach was proposed to explain the classification decisions of rectifier neural networks. In this work, we evaluate the discriminativeness of the generated explanations and analyze the theoretical foundation of LRP, i.e. Deep Taylor Decomposition. The experiments and analysis conclude that the explanations generated by LRP are not class-discriminative. Based on LRP, we propose Contrastive Layer-wise Relevance Propagation (CLRP), which is capable of producing instance-specific, class-discriminative, pixel-wise explanations. In the experiments, we use the CLRP to explain the decisions and understand the difference between neurons in individual classification decisions. We also evaluate the explanations quantitatively with a Pointing Game and an ablation study. Both qualitative and quantitative evaluations show that the CLRP generates better explanations than the LRP. The code is available.

arXiv comment16 pages, 5 figures, ACCV

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Striving for Simplicity: The All Convolutional Net
• Learning Important Features Through Propagating Activation Differences
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Object Detectors Emerge in Deep Scene CNNs
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Investigating the influence of noise and distractors on the interpretation of neural networks
• Understanding Deep Image Representations by Inverting Them
• Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance
• Understanding and Comparing Deep Neural Networks for Age and Gender Classification