Streamlining models with explanations in the learning loop

Abstract

Several explainable AI methods allow a Machine Learning user to get insights on the classification process of a black-box model in the form of local linear explanations. With such information, the user can judge which features are locally relevant for the classification outcome, and get an understanding of how the model reasons. Standard supervised learning processes are purely driven by the original features and target labels, without any feedback loop informed by the local relevance of the features identified by the post-hoc explanations. In this paper, we exploit this newly obtained information to design a feature engineering phase, where we combine explanations with feature values. To do so, we develop two different strategies, named Iterative Dataset Weighting and Targeted Replacement Values, which generate streamlined models that better mimic the explanation process presented to the user. We show how these streamlined models compare to the original black-box classifiers, in terms of accuracy and compactness of the newly produced explanations.

arXiv comment16 pages, 10 figures, available repository

References in corpus (12)

• SMOTE: Synthetic Minority Over-sampling Technique
• Towards Explainable Artificial Intelligence
• From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Explaining by Removing: A Unified Framework for Model Explanation
• WT5?! Training Text-to-Text Models to Explain their Predictions
• To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
• When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data
• An empirical study of the effect of background data size on the stability of SHapley Additive exPlanations (SHAP) for deep learning models
• Reflective-Net: Learning from Explanations
• Can Explanations Be Useful for Calibrating Black Box Models?
• Finding and Removing Clever Hans: Using Explanation Methods to Debug and Improve Deep Models