A Survey of Privacy Attacks in Machine Learning

Abstract

As machine learning becomes more widely used, the need to study its implications in security and privacy becomes more urgent. Although the body of work in privacy has been steadily growing over the past few years, research on the privacy aspects of machine learning has received less focus than the security aspects. Our contribution in this research is an analysis of more than 40 papers related to privacy attacks against machine learning that have been published during the past seven years. We propose an attack taxonomy, together with a threat model that allows the categorization of different attacks based on the adversarial knowledge, and the assets under attack. An initial exploration of the causes of privacy leaks is presented, as well as a detailed analysis of the different attacks. Finally, we present an overview of the most commonly proposed defenses and a discussion of the open problems and future directions identified during our analysis.

arXiv commentEdit to add DOI. Accepted in ACM Computing Surveys, please cite the journal version

References in corpus (27)

• Adam: A Method for Stochastic Optimization
• XGBoost: A Scalable Tree Boosting System
• Auto-Encoding Variational Bayes
• Deep Learning with Differential Privacy
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Understanding Neural Networks Through Deep Visualization
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Learning Differentially Private Recurrent Language Models
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Pointer Sentinel Mixture Models
• GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
• Algorithms that Remember: Model Inversion Attacks and Data Protection Law
• Privacy Risks of Securing Machine Learning Models against Adversarial Examples
• Understanding Membership Inferences on Well-Generalized Learning Models
• Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data
• White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• Overlearning Reveals Sensitive Attributes
• You are Who You Know and How You Behave: Attribute Inference Attacks via Users' Social Friends and Behaviors
• Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
• Towards Reverse-Engineering Black-Box Neural Networks

Cited by in corpus (20)

• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape
• Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies
• Machine Learning with Confidential Computing: A Systematization of Knowledge
• Differential privacy for medical deep learning: methods, tradeoffs, and deployment implications
• FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning
• AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Models
• Federated learning in food research
• Robustness and Cybersecurity in the EU Artificial Intelligence Act
• Optimal Private Median Estimation under Minimal Distributional Assumptions
• Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey
• R-GAP: Recursive Gradient Attack on Privacy
• Survey on AI Ethics: A Socio-technical Perspective
• A Comprehensive Study of Shapley Value in Data Analytics
• Omni: Automated Ensemble with Unexpected Models against Adversarial Evasion Attack
• A Human-Centered Privacy Approach (HCP) to AI
• Poisoning Attacks to Local Differential Privacy for Ranking Estimation
• Information Density Bounds for Privacy
• Short paper: Models in the dark -- Rectification and erasure under GDPR in ML supply chains
• Rethinking Client-oriented Federated Graph Learning