Visualizing Deep Neural Network Decisions: Prediction Difference Analysis

Abstract

This article presents the prediction difference analysis method for visualizing the response of a deep neural network to a specific input. When classifying images, the method highlights areas in a given input image that provide evidence for or against a certain class. It overcomes several shortcoming of previous methods and provides great additional insight into the decision making process of classifiers. Making neural network decisions interpretable through visualization is important both to improve models and to accelerate the adoption of black-box classifiers in application areas such as medicine. We illustrate the method in experiments on natural images (ImageNet data), as well as medical images (MRI brain scans).

ICLR2017

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Understanding Neural Networks Through Deep Visualization

Cited by in corpus (26)

• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Towards Explainable Artificial Intelligence
• Post-hoc explanation of black-box classifiers using confident itemsets
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
• Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output
• Visual pathways from the perspective of cost functions and multi-task deep neural networks
• Interpretable deep learning for nuclear deformation in heavy ion collisions
• Improving Interpretability of Deep Neural Networks with Semantic Information
• Self-explanatory Deep Salient Object Detection
• Relating Input Concepts to Convolutional Neural Network Decisions
• Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents
• Dissecting Catastrophic Forgetting in Continual Learning by Deep Visualization
• Towards Interrogating Discriminative Machine Learning Models
• SISC: End-to-end Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells
• TSInsight: A local-global attribution framework for interpretability in time-series data
• Human Understandable Explanation Extraction for Black-box Classification Models Based on Matrix Factorization
• Evaluating Explanation Methods for Neural Machine Translation
• Capacity allocation analysis of neural networks: A tool for principled architecture design
• Efficient Image Evidence Analysis of CNN Classification Results
• Using KL-divergence to focus Deep Visual Explanation
• Decision Explanation and Feature Importance for Invertible Networks
• Modeling Latent Attention Within Neural Networks
• Capacity allocation through neural network layers
• Deep Relevance Regularization: Interpretable and Robust Tumor Typing of Imaging Mass Spectrometry Data
• Explainable Deep Modeling of Tabular Data using TableGraphNet