Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation

Abstract

Deep neural networks have demonstrated remarkable performance in many data-driven and prediction-oriented applications, and sometimes even perform better than humans. However, their most significant drawback is the lack of interpretability, which makes them less attractive in many real-world applications. When relating to the moral problem or the environmental factors that are uncertain such as crime judgment, financial analysis, and medical diagnosis, it is essential to mine the evidence for the model's prediction (interpret model knowledge) to convince humans. Thus, investigating how to interpret model knowledge is of paramount importance for both academic research and real applications.

References in corpus (14)

• A Unified Approach to Interpreting Model Predictions
• Learning Important Features Through Propagating Activation Differences
• Interpretable Explanations of Black Boxes by Meaningful Perturbation
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Understanding Global Feature Contributions With Additive Importance Measures
• Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
• Problems with Shapley-value-based explanations as feature importance measures
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Gradients of Counterfactuals
• CXPlain: Causal Explanations for Model Interpretation under Uncertainty
• Neuron Shapley: Discovering the Responsible Neurons
• Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning
• Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
• Knowledge Consistency between Neural Networks and Beyond