Fooling Partial Dependence via Data Poisoning

Abstract

Many methods have been developed to understand complex predictive models and high expectations are placed on post-hoc model explainability. It turns out that such explanations are not robust nor trustworthy, and they can be fooled. This paper presents techniques for attacking Partial Dependence (plots, profiles, PDP), which are among the most popular methods of explaining any predictive model trained on tabular data. We showcase that PD can be manipulated in an adversarial manner, which is alarming, especially in financial or medical applications where auditability became a must-have trait supporting black-box machine learning. The fooling is performed via poisoning the data to bend and shift explanations in the desired direction using genetic and gradient algorithms. We believe this to be the first work using a genetic algorithm for manipulating explanations, which is transferable as it generalizes both ways: in a model-agnostic and an explanation-agnostic manner.

Accepted at ECML PKDD 2022

References in corpus (9)

• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• InterpretML: A Unified Framework for Machine Learning Interpretability
• Feature relevance quantification in explainable AI: A causal problem
• Fooling Neural Network Interpretations via Adversarial Model Manipulation
• dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python
• Proper Network Interpretability Helps Adversarial Robustness in Classification
• Smoothed Geometry for Robust Attribution
• The Bouncer Problem: Challenges to Remote Explainability
• Aggregating explanation methods for stable and robust explainability

Cited by in corpus (3)

• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• On the Robustness of Global Feature Effect Explanations
• Do not explain without context: addressing the blind spot of model explanations