Global Explanations of Neural Networks: Mapping the Landscape of Predictions

Abstract

A barrier to the wider adoption of neural networks is their lack of interpretability. While local explanation methods exist for one prediction, most global attributions still reduce neural network decisions to a single set of features. In response, we present an approach for generating global attributions called GAM, which explains the landscape of neural network predictions across subpopulations. GAM augments global explanations with the proportion of samples that each attribution best explains and specifies which samples are described by each attribution. Global explanations also have tunable granularity to detect more or fewer subpopulations. We demonstrate that GAM's global explanations 1) yield the known feature importances of simulated data, 2) match feature weights of interpretable statistical models on real data, and 3) are intuitive to practitioners through user studies. With more transparent predictions, GAM can help ensure neural network decisions are generated for the right reasons.

arXiv commentpublished at ACM/AAAI AIES

References in corpus (8)

• Towards A Rigorous Science of Interpretable Machine Learning
• Learning Important Features Through Propagating Activation Differences
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Distilling a Neural Network Into a Soft Decision Tree
• The (Un)reliability of saliency methods
• Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers
• Global Model Interpretation via Recursive Partitioning

Cited by in corpus (5)

• Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
• Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
• Considerations When Learning Additive Explanations for Black-Box Models
• Feature construction using explanations of individual predictions
• Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models