Towards Probabilistic Verification of Machine Unlearning

Abstract

The right to be forgotten, also known as the right to erasure, is the right of individuals to have their data erased from an entity storing it. The status of this long held notion was legally solidified recently by the General Data Protection Regulation (GDPR) in the European Union. Consequently, there is a need for mechanisms whereby users can verify if service providers comply with their deletion requests. In this work, we take the first step in proposing a formal framework to study the design of such verification mechanisms for data deletion requests -- also known as machine unlearning -- in the context of systems that provide machine learning as a service (MLaaS). Our framework allows the rigorous quantification of any verification mechanism based on standard hypothesis testing. Furthermore, we propose a novel backdoor-based verification mechanism and demonstrate its effectiveness in certifying data deletion with high confidence, thus providing a basis for quantitatively inferring machine unlearning. We evaluate our approach over a range of network architectures such as multi-layer perceptrons (MLP), convolutional neural networks (CNN), residual networks (ResNet), and long short-term memory (LSTM), as well as over 5 different datasets. We demonstrate that our approach has minimal effect on the ML service's accuracy but provides high confidence verification of unlearning. Our proposed mechanism works even if only a handful of users employ our system to ascertain compliance with data deletion requests. In particular, with just 5% of users participating, modifying half their data with a backdoor, and with merely 30 test queries, our verification mechanism has both false positive and false negative ratios below $10^{-3}$. We also show the effectiveness of our approach by testing it against an adaptive adversary that uses a state-of-the-art backdoor defense method.

code is available at https://github.com/inspire-group/unlearning-verification

References in corpus (10)

• Deep Learning with Differential Privacy
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Spectral Signatures in Backdoor Attacks
• LEAF: A Benchmark for Federated Settings
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Descent-to-Delete: Gradient-Based Methods for Machine Unlearning
• Label-Consistent Backdoor Attacks
• Radioactive data: tracing through training
• Certified Data Removal from Machine Learning Models

Cited by in corpus (13)

• Fast Yet Effective Machine Unlearning
• When Machine Unlearning Jeopardizes Privacy
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Remember What You Want to Forget: Algorithms for Machine Unlearning
• Landing AI on Networks: An equipment vendor viewpoint on Autonomous Driving Networks
• SoK: Machine Learning Governance
• Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
• Bayesian Inference Forgetting
• DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
• Federated Unlearning
• Lightweight machine unlearning in neural network
• 10 Security and Privacy Problems in Large Foundation Models
• TRAPDOOR: Repurposing backdoors to detect dataset bias in machine learning-based genomic analysis