Concept Whitening for Interpretable Image Recognition

Abstract

What does a neural network encode about a concept as we traverse through the layers? Interpretability in machine learning is undoubtedly important, but the calculations of neural networks are very challenging to understand. Attempts to see inside their hidden layers can either be misleading, unusable, or rely on the latent space to possess properties that it may not have. In this work, rather than attempting to analyze a neural network posthoc, we introduce a mechanism, called concept whitening (CW), to alter a given layer of the network to allow us to better understand the computation leading up to that layer. When a concept whitening module is added to a CNN, the axes of the latent space are aligned with known concepts of interest. By experiment, we show that CW can provide us a much clearer understanding for how the network gradually learns concepts over layers. CW is an alternative to a batch normalization layer in that it normalizes, and also decorrelates (whitens) the latent space. CW can be used in any layer of the network without hurting predictive performance.

arXiv commentAuthors' pre-publication version of a 2020 Nature Machine Intelligence article

References in corpus (29)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Recurrent Models of Visual Attention
• SmoothGrad: removing noise by adding noise
• Object Detectors Emerge in Deep Scene CNNs
• Multiple Object Recognition with Visual Attention
• Sanity Checks for Saliency Maps
• This Looks Like That: Deep Learning for Interpretable Image Recognition
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Concept Whitening for Interpretable Image Recognition
• Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
• On orthogonality and learning recurrent networks with long term dependencies
• Attention for Fine-Grained Categorization
• Full-Capacity Unitary Recurrent Neural Networks
• Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
• Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group
• Natural Neural Networks
• On Completeness-aware Concept-Based Explanations in Deep Neural Networks
• The Convolutional Tsetlin Machine
• Decorrelated Batch Normalization
• Generalized BackPropagation, Étude De Cas: Orthogonality
• Saccader: Improving Accuracy of Hard Attention Models for Vision
• Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• Whitening and Coloring batch transform for GANs
• EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction
• AOGNets: Compositional Grammatical Architectures for Deep Learning

Cited by in corpus (41)

• A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts
• Concept Whitening for Interpretable Image Recognition
• Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysis
• Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• Acquisition of Chess Knowledge in AlphaZero
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Entropy-based Logic Explanations of Neural Networks
• Normalization Techniques in Training DNNs: Methodology, Analysis and Application
• The Free Energy Principle for Perception and Action: A Deep Learning Perspective
• Explainable Machine Learning with Prior Knowledge: An Overview
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Promises and Pitfalls of Black-Box Concept Learning Models
• Concept-based Explainable Artificial Intelligence: A Survey
• Ethical Implementation of Artificial Intelligence to Select Embryos in In Vitro Fertilization
• IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammography
• GCExplainer: Human-in-the-Loop Concept-based Explanations for Graph Neural Networks
• Evaluating the Stability of Semantic Concept Representations in CNNs for Robust Explainability
• Emergence of Concepts in DNNs?
• A Comprehensive Survey on Self-Interpretable Neural Networks
• Memory Wrap: a Data-Efficient and Interpretable Extension to Image Classification Models
• CAT: Interpretable Concept-based Taylor Additive Models
• Towards Robust Metrics for Concept Representation Evaluation
• Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs
• Survey on AI Ethics: A Socio-technical Perspective
• Unveiling Molecular Moieties through Hierarchical Grad-CAM Graph Explainability
• Toward a Unified Framework for Debugging Concept-based Models
• Achieving Explainability for Plant Disease Classification with Disentangled Variational Autoencoders
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces
• The Definitions of Interpretability and Learning of Interpretable Models
• Interpreting Deep Neural Network-Based Receiver Under Varying Signal-To-Noise Ratios
• Interpretable by Design: Learning Predictors by Composing Interpretable Queries
• Debiasing Convolutional Neural Networks via Meta Orthogonalization
• Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis
• Cause and Effect: Hierarchical Concept-based Explanation of Neural Networks
• Interpretable Face Manipulation Detection via Feature Whitening
• Interpretable Deep Learning for Stock Returns: A Consensus-Bottleneck Asset Pricing Model
• A Framework to Learn with Interpretation
• Error Autocorrelation Objective Function for Improved System Modeling
• Corpus-level and Concept-based Explanations for Interpretable Document Classification
• How Faithful are Self-Explainable GNNs?