AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Models

Abstract

Explainable Artificial Intelligence (XAI) aims to uncover the decision-making processes of AI models. However, the data used for such explanations can pose security and privacy risks. Existing literature identifies attacks on machine learning models, including membership inference, model inversion, and model extraction attacks. These attacks target either the model or the training data, depending on the settings and parties involved. XAI tools can increase the vulnerability of model extraction attacks, which is a concern when model owners prefer black-box access, thereby keeping model parameters and architecture private. To exploit this risk, we propose AUTOLYCUS, a novel retraining (learning) based model extraction attack framework against interpretable models under black-box settings. As XAI tools, we exploit Local Interpretable Model-Agnostic Explanations (LIME) and Shapley values (SHAP) to infer decision boundaries and create surrogate models that replicate the functionality of the target model. LIME and SHAP are mainly chosen for their realistic yet information-rich explanations, coupled with their extensive adoption, simplicity, and usability. We evaluate AUTOLYCUS on six machine learning datasets, measuring the accuracy and similarity of the surrogate model to the target model. The results show that AUTOLYCUS is highly effective, requiring significantly fewer queries compared to state-of-the-art attacks, while maintaining comparable accuracy and similarity. We validate its performance and transferability on multiple interpretable ML models, including decision trees, logistic regression, naive bayes, and k-nearest neighbor. Additionally, we show the resilience of AUTOLYCUS against proposed countermeasures.

This work is published in the Proceedings on Privacy Enhancing Technologies (PoPETs), Vol. 2024, Issue 4, 2024

References in corpus (12)

• Scikit-learn: Machine Learning in Python
• Axiomatic Attribution for Deep Networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Learning how to explain neural networks: PatternNet and PatternAttribution
• A Survey of Privacy Attacks in Machine Learning
• Agnostic Active Learning Without Constraints
• Stealing Neural Networks via Timing Side Channels
• Model extraction from counterfactual explanations
• MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI

Cited by in corpus (1)

• Towards integration of Privacy Enhancing Technologies in Explainable Artificial Intelligence