99 citations · 222 across the 29 of their papers we have counts for
10 papers · 1 filter
Batch Clipping and Adaptive Layerwise Clipping for Differential Private Stochastic Gradient Descent
Toan N. Nguyen, Phuong Ha Nguyen, Lam M. Nguyen +1
Each round in Differential Private Stochastic Gradient Descent (DPSGD) transmits a sum of clipped gradients obfuscated with Gaussian noise to a central server which uses this to up…
Considerations on the Theory of Training Models with Differential Privacy
Marten van Dijk, Phuong Ha Nguyen
In federated learning collaborative learning takes place by a set of clients who each want to remain in control of how their local training data is used, in particular, how can eac…
Generalizing DP-SGD with Shuffling and Batch Clipping
Marten van Dijk, Phuong Ha Nguyen, Toan N. Nguyen +1
Classical differential private DP-SGD implements individual clipping with random subsampling, which forces a mini-batch SGD approach. We provide a general differential private algo…
Finite-Sum Optimization: A New Perspective for Convergence to a Global Solution
Lam M. Nguyen, Trang H. Tran, Marten van Dijk
Deep neural networks (DNNs) have shown great success in many machine learning tasks. Their training is challenging since the loss surface of the network architecture is generally n…
Proactive DP: A Multple Target Optimization Framework for DP-SGD
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +2
We introduce a multiple target optimization framework for DP-SGD referred to as pro-active DP. In contrast to traditional DP accountants, which are used to track the expenditure of…
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +3
Hogwild! implements asynchronous Stochastic Gradient Descent (SGD) where multiple threads in parallel access a common repository containing training data, perform SGD iterations an…