Model extraction from counterfactual explanations

Abstract

Post-hoc explanation techniques refer to a posteriori methods that can be used to explain how black-box machine learning models produce their outcomes. Among post-hoc explanation techniques, counterfactual explanations are becoming one of the most popular methods to achieve this objective. In particular, in addition to highlighting the most important features used by the black-box model, they provide users with actionable explanations in the form of data instances that would have received a different outcome. Nonetheless, by doing so, they also leak non-trivial information about the model itself, which raises privacy issues. In this work, we demonstrate how an adversary can leverage the information provided by counterfactual explanations to build high-fidelity and high-accuracy model extraction attacks. More precisely, our attack enables the adversary to build a faithful copy of a target model by accessing its counterfactual explanations. The empirical evaluation of the proposed attack on black-box models trained on real-world datasets demonstrates that they can achieve high-fidelity and high-accuracy extraction even under low query budgets.

References in corpus (23)

• Artificial Intelligence: the global landscape of ethics guidelines
• Methods for Interpreting and Understanding Deep Neural Networks
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• How to Explain Individual Classification Decisions
• Sanity Checks for Saliency Maps
• Actionable Recourse in Linear Classification
• Adversarial Frontier Stitching for Remote Neural Network Watermarking
• Supersparse Linear Integer Models for Optimized Medical Scoring Systems
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data
• Explanations can be manipulated and geometry is to blame
• Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• Boolean Decision Rules via Column Generation
• Inverse Classification for Comparison-based Interpretability in Machine Learning
• Entangled Watermarks as a Defense against Model Extraction
• A framework for the extraction of Deep Neural Networks by leveraging public data
• CSI Neural Network: Using Side-channels to Recover Your Artificial Neural Network Information
• The Bouncer Problem: Challenges to Remote Explainability
• Learning Fair Rule Lists
• Extraction of Complex DNN Models: Real Threat or Boogeyman?

Cited by in corpus (6)

• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Models
• MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI
• Seven challenges for harmonizing explainability requirements
• Supervised Feature Compression based on Counterfactual Analysis
• Characterizing the risk of fairwashing