Interpreting CNNs via Decision Trees

Abstract

This paper aims to quantitatively explain rationales of each prediction that is made by a pre-trained convolutional neural network (CNN). We propose to learn a decision tree, which clarifies the specific reason for each prediction made by the CNN at the semantic level. I.e., the decision tree decomposes feature representations in high conv-layers of the CNN into elementary concepts of object parts. In this way, the decision tree tells people which object parts activate which filters for the prediction and how much they contribute to the prediction score. Such semantic and quantitative explanations for CNN predictions have specific values beyond the traditional pixel-level analysis of CNNs. More specifically, our method mines all potential decision modes of the CNN, where each mode represents a common case of how the CNN uses object parts for prediction. The decision tree organizes all potential decision modes in a coarse-to-fine manner to explain CNN predictions at different fine-grained levels. Experiments have demonstrated the effectiveness of the proposed method.

References in corpus (11)

• Intriguing properties of neural networks
• How transferable are features in deep neural networks?
• Towards A Rigorous Science of Interpretable Machine Learning
• Understanding Black-box Predictions via Influence Functions
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Distilling a Neural Network Into a Soft Decision Tree
• Explainable Neural Networks based on Additive Index Models
• Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning
• Network Transplanting

Cited by in corpus (17)

• Concise Fuzzy System Modeling Integrating Soft Subspace Clustering and Sparse Learning
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• Visual Interpretability for Deep Learning: a Survey
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policy
• Improving the Interpretability of Deep Neural Networks with Knowledge Distillation
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Explaining Neural Networks Semantically and Quantitatively
• Architecting Dependable Learning-enabled Autonomous Systems: A Survey
• Facial age estimation by deep residual decision making
• Interpretable Visual Reasoning via Induced Symbolic Space
• Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
• Explanatory Graphs for CNNs
• Mining Deep And-Or Object Structures via Cost-Sensitive Question-Answer-Based Active Annotations
• Progress in deep Markov State Modeling: Coarse graining and experimental data restraints
• Extracting Interpretable Concept-Based Decision Trees from CNNs
• Leveraging Model Interpretability and Stability to increase Model Robustness