17 citations · 36 across the 12 of their papers we have counts for
10 papers · 1 filter
PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization
Murat Bilgehan Ertan, Xiaochen Zhu, Phuong Ha Nguyen +2
We introduce PACZero, a family of PAC-private zeroth-order mechanisms for fine-tuning large language models that delivers usable utility at . This privacy regime…
Batch Clipping and Adaptive Layerwise Clipping for Differential Private Stochastic Gradient Descent
Toan N. Nguyen, Phuong Ha Nguyen, Lam M. Nguyen +1
Each round in Differential Private Stochastic Gradient Descent (DPSGD) transmits a sum of clipped gradients obfuscated with Gaussian noise to a central server which uses this to up…
Considerations on the Theory of Training Models with Differential Privacy
Marten van Dijk, Phuong Ha Nguyen
In federated learning collaborative learning takes place by a set of clients who each want to remain in control of how their local training data is used, in particular, how can eac…
Generalizing DP-SGD with Shuffling and Batch Clipping
Marten van Dijk, Phuong Ha Nguyen, Toan N. Nguyen +1
Classical differential private DP-SGD implements individual clipping with random subsampling, which forces a mini-batch SGD approach. We provide a general differential private algo…
Proactive DP: A Multple Target Optimization Framework for DP-SGD
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +2
We introduce a multiple target optimization framework for DP-SGD referred to as pro-active DP. In contrast to traditional DP accountants, which are used to track the expenditure of…
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +3
Hogwild! implements asynchronous Stochastic Gradient Descent (SGD) where multiple threads in parallel access a common repository containing training data, perform SGD iterations an…