Fairwashing Explanations with Off-Manifold Detergent

Abstract

Explanation methods promise to make black-box classifiers more transparent. As a result, it is hoped that they can act as proof for a sensible, fair and trustworthy decision-making process of the algorithm and thereby increase its acceptance by the end-users. In this paper, we show both theoretically and experimentally that these hopes are presently unfounded. Specifically, we show that, for any classifier $g$, one can always construct another classifier $\tilde{g}$ which has the same behavior on the data (same train, validation, and test error) but has arbitrarily manipulated explanation maps. We derive this statement theoretically using differential geometry and demonstrate it experimentally for various explanation methods, architectures, and datasets. Motivated by our theoretical insights, we then propose a modification of existing explanation methods which makes them significantly more robust.

22 pages with 43 figures, to be published in ICML2020

Cited by in corpus (10)

• Explainable Deep One-Class Classification
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Shapley explainability on the data manifold
• PredDiff: Explanations and Interactions from Conditional Expectations
• A Survey on the Robustness of Feature Importance and Counterfactual Explanations
• Towards Robust Explanations for Deep Neural Networks
• Counterfactual Explanations Can Be Manipulated
• Seven challenges for harmonizing explainability requirements
• Selective Ensembles for Consistent Predictions
• A Sociotechnical View of Algorithmic Fairness