Visualizing Deep Neural Network Decisions: Prediction Difference Analysis

Abstract

This article presents the prediction difference analysis method for visualizing the response of a deep neural network to a specific input. When classifying images, the method highlights areas in a given input image that provide evidence for or against a certain class. It overcomes several shortcoming of previous methods and provides great additional insight into the decision making process of classifiers. Making neural network decisions interpretable through visualization is important both to improve models and to accelerate the adoption of black-box classifiers in application areas such as medicine. We illustrate the method in experiments on natural images (ImageNet data), as well as medical images (MRI brain scans).

ICLR2017

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Understanding Neural Networks Through Deep Visualization

Cited by in corpus (17)

• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
• Visual pathways from the perspective of cost functions and multi-task deep neural networks
• Interpretable deep learning for nuclear deformation in heavy ion collisions
• Improving Interpretability of Deep Neural Networks with Semantic Information
• Self-explanatory Deep Salient Object Detection
• Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents
• Relating Input Concepts to Convolutional Neural Network Decisions
• SISC: End-to-end Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells
• Towards Interrogating Discriminative Machine Learning Models
• Human Understandable Explanation Extraction for Black-box Classification Models Based on Matrix Factorization
• Efficient Image Evidence Analysis of CNN Classification Results
• Capacity allocation analysis of neural networks: A tool for principled architecture design
• Using KL-divergence to focus Deep Visual Explanation
• Capacity allocation through neural network layers
• Modeling Latent Attention Within Neural Networks