AdaCliP: Adaptive Clipping for Private SGD
arXiv:1908.07643
Abstract
Privacy preserving machine learning algorithms are crucial for learning models over user data to protect sensitive information. Motivated by this, differentially private stochastic gradient descent (SGD) algorithms for training machine learning models have been proposed. At each step, these algorithms modify the gradients and add noise proportional to the sensitivity of the modified gradients. Under this framework, we propose AdaCliP, a theoretically motivated differentially private SGD algorithm that provably adds less noise compared to the previous methods, by using coordinate-wise adaptive clipping of the gradient. We empirically demonstrate that AdaCliP reduces the amount of added noise and produces models with better accuracy.
References in corpus (7)
- Deep Learning with Differential Privacy
- Federated Learning for Mobile Keyboard Prediction
- Differentially Private Learning with Adaptive Clipping
- Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
- Differentially Private Releasing via Deep Generative Model (Technical Report)
- Three Tools for Practical Differential Privacy
- Optimal Noise-Adding Mechanism in Additive Differential Privacy
Cited by in corpus (22)
- Differentially Private Learning with Adaptive Clipping
- Large Language Models Can Be Strong Differentially Private Learners
- Label Leakage and Protection in Two-party Split Learning
- Understanding Gradient Clipping in Private SGD: A Geometric Perspective
- DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation
- Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learning
- Practical and Private (Deep) Learning without Sampling or Shuffling
- DPIS: An Enhanced Mechanism for Differentially Private SGD with Importance Sampling
- Fast-adapting and Privacy-preserving Federated Recommender System
- Differentially Private Accelerated Optimization Algorithms
- Instance-optimal Mean Estimation Under Differential Privacy
- Locally Differentially Private Analysis of Graph Statistics
- Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent
- Fast Dimension Independent Private AdaGrad on Publicly Estimated Subspaces
- Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy
- Dynamic Differential-Privacy Preserving SGD
- Dynamic Global Sensitivity for Differentially Private Contextual Bandits
- Adaptive Differentially Private Empirical Risk Minimization
- Universal Private Estimators
- N-grams Bayesian Differential Privacy
- Efficient Hyperparameter Optimization for Differentially Private Deep Learning
- The Relative Gaussian Mechanism and its Application to Private Gradient Descent