Algorithms that Remember: Model Inversion Attacks and Data Protection Law

Abstract

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around `model inversion' and `membership inference' attacks, which indicate that the process of turning training data into machine learned systems is not one-way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.

arXiv comment15 pages, 1 figure

References in corpus (6)

• Stealing Machine Learning Models via Prediction APIs
• Differentially Private Empirical Risk Minimization
• Differential Privacy and Machine Learning: a Survey and Review
• Third Party Tracking in the Mobile Ecosystem
• Decision Provenance: Harnessing data flow for accountable systems
• On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches

Cited by in corpus (23)

• Challenges in Deploying Machine Learning: a Survey of Case Studies
• A Survey of Privacy Attacks in Machine Learning
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• SoK: Exploring the State of the Art and the Future Potential of Artificial Intelligence in Digital Forensic Investigation
• Making AI Forget You: Data Deletion in Machine Learning
• Security and Privacy Issues in Deep Learning
• Adaptive Machine Unlearning
• SoK: Realistic Adversarial Attacks and Defenses for Intelligent Network Intrusion Detection
• Bias in Data-driven AI Systems -- An Introductory Survey
• Knowledge Cross-Distillation for Membership Privacy
• Anonymizing Machine Learning Models
• Enhanced Membership Inference Attacks against Machine Learning Models
• Towards Security Threats of Deep Learning Systems: A Survey
• Privacy Attacks on Network Embeddings
• Data and Model Dependencies of Membership Inference Attack
• Mysterious and Manipulative Black Boxes: A Qualitative Analysis of Perceptions on Recommender Systems
• Promoting Distributed Trust in Machine Learning and Computational Simulation via a Blockchain Network
• Correlation inference attacks against machine learning models
• Machine Learning Models Disclosure from Trusted Research Environments (TRE), Challenges and Opportunities
• Marketplace for AI Models
• A Human-Centered Privacy Approach (HCP) to AI
• Reducing Risk of Model Inversion Using Privacy-Guided Training
• Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges