Towards Rigorous Interpretations: a Formalisation of Feature Attribution

Abstract

Feature attribution is often loosely presented as the process of selecting a subset of relevant features as a rationale of a prediction. Task-dependent by nature, precise definitions of "relevance" encountered in the literature are however not always consistent. This lack of clarity stems from the fact that we usually do not have access to any notion of ground-truth attribution and from a more general debate on what good interpretations are. In this paper we propose to formalise feature selection/attribution based on the concept of relaxed functional dependence. In particular, we extend our notions to the instance-wise setting and derive necessary properties for candidate selection solutions, while leaving room for task-dependence. By computing ground-truth attributions on synthetic datasets, we evaluate many state-of-the-art attribution methods and show that, even when optimised, some fail to verify the proposed properties and provide wrong solutions.

arXiv comment38th International Conference on Machine Learning (ICML 2021)

References in corpus (12)

• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Predictive learning via rule ensembles
• SmoothGrad: removing noise by adding noise
• Sanity Checks for Saliency Maps
• The (Un)reliability of saliency methods
• Problems with Shapley-value-based explanations as feature importance measures
• Feature Selection using Stochastic Gates
• Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction
• Explaining Deep Neural Networks
• Feature Removal Is a Unifying Principle for Model Explanation Methods
• Challenging common interpretability assumptions in feature attribution explanations

Cited by in corpus (2)

• Impossibility Theorems for Feature Attribution
• Explainability as statistical inference