Understanding Global Feature Contributions With Additive Importance Measures

Abstract

Understanding the inner workings of complex machine learning models is a long-standing problem and most recent research has focused on local interpretability. To assess the role of individual input features in a global sense, we explore the perspective of defining feature importance through the predictive power associated with each feature. We introduce two notions of predictive power (model-based and universal) and formalize this approach with a framework of additive importance measures, which unifies numerous methods in the literature. We then propose SAGE, a model-agnostic method that quantifies predictive power while accounting for feature interactions. Our experiments show that SAGE can be calculated efficiently and that it assigns more accurate importance values than other methods.

NeurIPS 2020 Camera Ready

References in corpus (4)

• Axiomatic Attribution for Deep Networks
• Feature relevance quantification in explainable AI: A causal problem
• L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data
• Sensitivity based Neural Networks Explanations

Cited by in corpus (40)

• The impact of feature importance methods on the interpretation of defect classifiers
• Explaining by Removing: A Unified Framework for Model Explanation
• Grouped Feature Importance and Combined Features Effect Plot
• Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams
• A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort study
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• Relative Feature Importance
• PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation
• MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks
• Feature Removal Is a Unifying Principle for Model Explanation Methods
• Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression
• Efficient nonparametric statistical inference on population feature importance using Shapley values
• Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation
• SHAFF: Fast and consistent SHApley eFfect estimates via random Forests
• Explaining a Deep Reinforcement Learning Docking Agent Using Linear Model Trees with User Adapted Visualization
• Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles
• Floodgate: inference for model-free variable importance
• Shapley explainability on the data manifold
• "You might think about slightly revising the title": identifying hedges in peer-tutoring interactions
• LLpowershap: Logistic Loss-based Automated Shapley Values Feature Selection Method
• Explaining the data or explaining a model? Shapley values that uncover non-linear dependencies
• Unifying local and global model explanations by functional decomposition of low dimensional structures
• Nonlinear blind source separation exploiting spatial nonstationarity
• Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
• WebSHAP: Towards Explaining Any Machine Learning Models Anywhere
• DU-Shapley: A Shapley Value Proxy for Efficient Dataset Valuation
• Interpretable Model Drift Detection
• DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability
• Defining and Quantifying the Emergence of Sparse Concepts in DNNs
• Why is "Problems" Predictive of Positive Sentiment? A Case Study of Explaining Unintuitive Features in Sentiment Classification
• Balancing the Norwegian regulated power market anno 2016 to 2022
• A Unified Game-Theoretic Interpretation of Adversarial Robustness
• FastSHAP: Real-Time Shapley Value Estimation
• Data-driven advice for interpreting local and global model predictions in bioinformatics problems
• Factor Importance Ranking and Selection using Total Indices
• EDDA: Explanation-driven Data Augmentation to Improve Explanation Faithfulness
• Decorrelated Variable Importance
• Using Shapley Values and Variational Autoencoders to Explain Predictive Models with Dependent Mixed Features
• Marginal Contribution Feature Importance -- an Axiomatic Approach for The Natural Case
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance