Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation

Abstract

Complex nonlinear models such as deep neural network (DNNs) have become an important tool for image classification, speech recognition, natural language processing, and many other fields of application. These models however lack transparency due to their complex nonlinear structure and to the complex data distributions to which they typically apply. As a result, it is difficult to fully characterize what makes these models reach a particular decision for a given input. This lack of transparency can be a drawback, especially in the context of sensitive applications such as medical analysis or security. In this short paper, we summarize a recent technique introduced by Bach et al. [1] that explains predictions by decomposing the classification decision of DNN models in terms of input variables.

arXiv commentPresented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems

References in corpus (4)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks

Cited by in corpus (10)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Explanations in Autonomous Driving: A Survey
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Gradual Channel Pruning while Training using Feature Relevance Scores for Convolutional Neural Networks
• Towards computational fluorescence microscopy: Machine learning-based integrated prediction of morphological and molecular tumor profiles
• Learning and Interpreting Multi-Multi-Instance Learning Networks
• Fibres of Failure: Classifying errors in predictive processes
• Relevant-features based Auxiliary Cells for Energy Efficient Detection of Natural Errors
• AdjointBackMapV2: Precise Reconstruction of Arbitrary CNN Unit's Activation via Adjoint Operators