Auditing Differentially Private Machine Learning: How Private is Private SGD?

Abstract

We investigate whether Differentially Private SGD offers better privacy in practice than what is guaranteed by its state-of-the-art analysis. We do so via novel data poisoning attacks, which we show correspond to realistic privacy attacks. While previous work (Ma et al., arXiv 2019) proposed this connection between differential privacy and data poisoning as a defense against data poisoning, our use as a tool for understanding the privacy of a specific mechanism is new. More generally, our work takes a quantitative, empirical approach to understanding the privacy afforded by specific implementations of differentially private algorithms that we believe has the potential to complement and influence analytical work on differential privacy.

References in corpus (6)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Gaussian Differential Privacy
• Rényi Differential Privacy of the Sampled Gaussian Mechanism
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• Data Poisoning against Differentially-Private Learners: Attacks and Defenses

Cited by in corpus (21)

• How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
• Privacy in Deep Learning: A Survey
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Differentially Private Learning Needs Better Features (or Much More Data)
• Practical and Private (Deep) Learning without Sampling or Shuffling
• Evaluations of Machine Learning Privacy Defenses are Misleading
• Enhanced Membership Inference Attacks against Machine Learning Models
• QuerySnout: Automating the Discovery of Attribute Inference Attacks against Query-Based Systems
• SoK: Machine Learning Governance
• Pufferfish Privacy: An Information-Theoretic Study
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
• On the Importance of Difficulty Calibration in Membership Inference Attacks
• Revealing the True Cost of Locally Differentially Private Protocols: An Auditing Perspective
• Where have you been? A Study of Privacy Risk for Point-of-Interest Recommendation
• QueryCheetah: Fast Automated Discovery of Attribute Inference Attacks Against Query-Based Systems
• Privacy in Practice: Private COVID-19 Detection in X-Ray Images (Extended Version)
• Differentially Private Computation of Basic Reproduction Numbers in Networked Epidemic Models
• Curator Attack: When Blackbox Differential Privacy Auditing Loses Its Power
• An Efficient DP-SGD Mechanism for Large Scale NLP Models
• Statistical Privacy Guarantees of Machine Learning Preprocessing Techniques
• Tensorization of neural networks for improved privacy and interpretability