How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy

Abstract

ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the privacy of ML training data. Differential Privacy (DP) has become a gold standard for making formal statements about data anonymization. However, while some adoption of DP has happened in industry, attempts to apply DP to real world complex ML models are still few and far between. The adoption of DP is hindered by limited practical guidance of what DP protection entails, what privacy guarantees to aim for, and the difficulty of achieving good privacy-utility-computation trade-offs for ML models. Tricks for tuning and maximizing performance are scattered among papers or stored in the heads of practitioners. Furthermore, the literature seems to present conflicting evidence on how and whether to apply architectural adjustments and which components are "safe" to use with DP. This work is a self-contained guide that gives an in-depth overview of the field of DP ML and presents information about achieving the best possible DP ML model with rigorous privacy guarantees. Our target audience is both researchers and practitioners. Researchers interested in DP for ML will benefit from a clear overview of current advances and areas for improvement. We include theory-focused sections that highlight important topics such as privacy accounting and its assumptions, and convergence. For a practitioner, we provide a background in DP theory and a clear step-by-step guide for choosing an appropriate privacy definition and approach, implementing DP training, potentially updating the model architecture, and tuning hyperparameters. For both researchers and practitioners, consistently and fully reporting privacy guarantees is critical, and so we propose a set of specific best practices for stating guarantees.

References in corpus (103)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning with Differential Privacy
• Practical Bayesian Optimization of Machine Learning Algorithms
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Renyi Differential Privacy
• Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
• Learning Differentially Private Recurrent Language Models
• Visualizing the Loss Landscape of Neural Nets
• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Differentially Private Empirical Risk Minimization
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Cascaded Diffusion Models for High Fidelity Image Generation
• Train longer, generalize better: closing the generalization gap in large batch training of neural networks
• Differentially Private Generative Adversarial Network
• Prochlo: Strong Privacy for Analytics in the Crowd
• A simple and practical algorithm for differentially private data release
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
• A Field Guide to Federated Optimization
• Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences
• Adaptive Bound Optimization for Online Convex Optimization
• A General Approach to Adding Differential Privacy to Iterative Training Procedures
• Privacy Amplification by Iteration
• White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
• A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learning
• Rényi Differential Privacy of the Sampled Gaussian Mechanism
• Differentially Private Learning with Adaptive Clipping
• Evaluating Differentially Private Machine Learning in Practice
• Private Stochastic Convex Optimization with Optimal Rates
• Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising
• Large Language Models Can Be Strong Differentially Private Learners
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• Differentially Private Learning Needs Better Features (or Much More Data)
• Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds
• Functional Mechanism: Regression Analysis under Differential Privacy
• Understanding Gradient Clipping in Private SGD: A Geometric Perspective
• Scaling Up Models and Data with $\text{t5x}$ and $\text{seqio}$
• Differentially Private Fine-tuning of Language Models
• Natural Language Understanding with Privacy-Preserving BERT
• Why are Adaptive Methods Good for Attention Models?
• The Skellam Mechanism for Differentially Private Federated Learning
• Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation
• Unlocking High-Accuracy Differentially Private Image Classification through Scale
• Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry
• Efficient Per-Example Gradient Computations
• Benchmarking Differentially Private Synthetic Data Generation Algorithms
• Opacus: User-Friendly Differential Privacy Library in PyTorch
• Practical and Private (Deep) Learning without Sampling or Shuffling
• New Oracle-Efficient Algorithms for Private Synthetic Data Release
• Characterizing signal propagation to close the performance gap in unnormalized ResNets
• Decision Tree Classification with Differential Privacy: A Survey
• Toward Training at ImageNet Scale with Differential Privacy
• On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
• The Large Margin Mechanism for Differentially Private Maximization
• Permute-and-Flip: A new mechanism for differentially private selection
• Reviewing and Improving the Gaussian Mechanism for Differential Privacy
• Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses
• The Tree Ensemble Layer: Differentiability meets Conditional Computation
• That which we call private
• Differentially Private Accelerated Optimization Algorithms
• Locally Private Bayesian Inference for Count Models
• Tight Auditing of Differentially Private Machine Learning
• Differentially private training of residual networks with scale normalisation
• Private Adaptive Gradient Methods for Convex Optimization
• Iterative Methods for Private Synthetic Data: Unifying Framework and New Methods
• Differentially Private Query Release Through Adaptive Projection
• Benefits and Pitfalls of the Exponential Mechanism with Applications to Hilbert Spaces and Functional PCA
• Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
• Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment
• Privacy Auditing with One (1) Training Run
• Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss
• AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data
• Privacy-Preserving Gradient Boosting Decision Trees
• On the Convergence and Calibration of Deep Learning with Differential Privacy
• Gradient Perturbation is Underrated for Differentially Private Convex Optimization
• Adversarial Learning of Privacy-Preserving and Task-Oriented Representations
• Differentially private inference via noisy optimization
• A General Framework for Auditing Differentially Private Machine Learning
• Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
• Hyperparameter Tuning with Renyi Differential Privacy
• One-shot Empirical Privacy Estimation for Federated Learning
• Unleashing the Power of Randomization in Auditing Differentially Private ML
• Bounding Training Data Reconstruction in Private (Deep) Learning
• Label differential privacy via clustering
• Does Label Differential Privacy Prevent Label Inference Attacks?
• Scalable and Provably Accurate Algorithms for Differentially Private Distributed Decision Tree Learning
• Private Adaptive Optimization with Side Information
• Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent
• Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano
• Node-Level Differentially Private Graph Neural Networks
• Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective
• Public Data-Assisted Mirror Descent for Private Model Training
• DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy
• Federated Learning of Gboard Language Models with Differential Privacy
• CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning
• Faster Rates of Convergence to Stationary Points in Differentially Private Optimization
• TAN Without a Burn: Scaling Laws of DP-SGD
• Differentially Private Image Classification from Features
• (Amplified) Banded Matrix Factorization: A unified approach to private training
• Beyond Uniform Lipschitz Condition in Differentially Private Optimization

Cited by in corpus (7)

• Advancing privacy in learning analytics using differential privacy
• Towards integration of Privacy Enhancing Technologies in Explainable Artificial Intelligence
• Synthetic Trajectory Generation Through Convolutional Neural Networks
• Federated Learning With Individualized Privacy Through Client Sampling
• Cyclic Adaptive Private Synthesis for Sharing Real-World Data in Education
• Differentially Private Synthetic Data Release for Topics API Outputs
• SynQP: A Framework and Metrics for Evaluating the Quality and Privacy Risk of Synthetic Data