Sampling, Intervention, Prediction, Aggregation: A Generalized Framework for Model-Agnostic Interpretations

Abstract

Model-agnostic interpretation techniques allow us to explain the behavior of any predictive model. Due to different notations and terminology, it is difficult to see how they are related. A unified view on these methods has been missing. We present the generalized SIPA (sampling, intervention, prediction, aggregation) framework of work stages for model-agnostic interpretations and demonstrate how several prominent methods for feature effects can be embedded into the proposed framework. Furthermore, we extend the framework to feature importance computations by pointing out how variance-based and performance-based importance measures are based on the same work stages. The SIPA framework reduces the diverse set of model-agnostic techniques to a single methodology and establishes a common terminology to discuss them in future work.

References in corpus (6)

• A Unified Approach to Interpreting Model Predictions
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models
• Visualizing the Feature Importance for Black Box Models
• A Simple and Effective Model-Based Variable Importance Measure
• Interpretation of Prediction Models Using the Input Gradient

Cited by in corpus (4)

• Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process
• Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach
• Grouped Feature Importance and Combined Features Effect Plot
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena