A Survey on the Robustness of Feature Importance and Counterfactual Explanations

Abstract

There exist several methods that aim to address the crucial task of understanding the behaviour of AI/ML models. Arguably, the most popular among them are local explanations that focus on investigating model behaviour for individual instances. Several methods have been proposed for local analysis, but relatively lesser effort has gone into understanding if the explanations are robust and accurately reflect the behaviour of underlying models. In this work, we present a survey of the works that analysed the robustness of two classes of local explanations (feature importance and counterfactual explanations) that are popularly used in analysing AI/ML models in finance. The survey aims to unify existing definitions of robustness, introduces a taxonomy to classify different robustness approaches, and discusses some interesting results. Finally, the survey introduces some pointers about extending current robustness analysis approaches so as to identify reliable explainability methods.

arXiv comment4 pages plus references. Accepted at the workshop on Explainable AI in Finance (XAI-FIN21). Camera-ready version. V2: Added more references and expanded robust explanations for counterfactuals

References in corpus (15)

• Axiomatic Attribution for Deep Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• Sanity Checks for Saliency Maps
• Learning Model-Agnostic Counterfactual Explanations for Tabular Data
• The (Un)reliability of saliency methods
• Explanations can be manipulated and geometry is to blame
• Fooling Neural Network Interpretations via Adversarial Model Manipulation
• Benchmarking Attribution Methods with Relative Feature Importance
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• Towards Robust and Reliable Algorithmic Recourse
• Fairwashing Explanations with Off-Manifold Detergent
• A Causal Perspective on Meaningful and Robust Algorithmic Recourse
• CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms
• Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
• Counterfactual Explanations Can Be Manipulated