17 citations · 20 across the 7 of their papers we have counts for
7 papers
FOGO: Forgetting-aware Orthogonalization Optimizer
Toan Nguyen, Yang Liu, Trung Le +2
We argue that forgetting is not confined to continual learning but is a general optimization phenomenon: during standard training, dominant mini-batch gradients suppress rare but u…
Batch Clipping and Adaptive Layerwise Clipping for Differential Private Stochastic Gradient Descent
Toan N. Nguyen, Phuong Ha Nguyen, Lam M. Nguyen +1
Each round in Differential Private Stochastic Gradient Descent (DPSGD) transmits a sum of clipped gradients obfuscated with Gaussian noise to a central server which uses this to up…
Generalizing DP-SGD with Shuffling and Batch Clipping
Marten van Dijk, Phuong Ha Nguyen, Toan N. Nguyen +1
Classical differential private DP-SGD implements individual clipping with random subsampling, which forces a mini-batch SGD approach. We provide a general differential private algo…
Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings
Zhen Yu, Toan Nguyen, Yaniv Gal +7
In practice, many medical datasets have an underlying taxonomy defined over the disease label space. However, existing classification algorithms for medical diagnoses often assume…
Proactive DP: A Multple Target Optimization Framework for DP-SGD
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +2
We introduce a multiple target optimization framework for DP-SGD referred to as pro-active DP. In contrast to traditional DP accountants, which are used to track the expenditure of…
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +3
Hogwild! implements asynchronous Stochastic Gradient Descent (SGD) where multiple threads in parallel access a common repository containing training data, perform SGD iterations an…
Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +3
The feasibility of federated learning is highly constrained by the server-clients infrastructure in terms of network communication. Most newly launched smartphones and IoT devices…