17 citations · 18 across the 5 of their papers we have counts for
7 papers
FOGO: Forgetting-aware Orthogonalization Optimizer
Toan Nguyen, Yang Liu, Trung Le +2
We argue that forgetting is not confined to continual learning but is a general optimization phenomenon: during standard training, dominant mini-batch gradients suppress rare but u…
Batch Clipping and Adaptive Layerwise Clipping for Differential Private Stochastic Gradient Descent
Toan N. Nguyen, Phuong Ha Nguyen, Lam M. Nguyen +1
Each round in Differential Private Stochastic Gradient Descent (DPSGD) transmits a sum of clipped gradients obfuscated with Gaussian noise to a central server which uses this to up…
Generalizing DP-SGD with Shuffling and Batch Clipping
Marten van Dijk, Phuong Ha Nguyen, Toan N. Nguyen +1
Classical differential private DP-SGD implements individual clipping with random subsampling, which forces a mini-batch SGD approach. We provide a general differential private algo…
Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings
Zhen Yu, Toan Nguyen, Yaniv Gal +7
In practice, many medical datasets have an underlying taxonomy defined over the disease label space. However, existing classification algorithms for medical diagnoses often assume…
Proactive DP: A Multple Target Optimization Framework for DP-SGD
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +2
We introduce a multiple target optimization framework for DP-SGD referred to as pro-active DP. In contrast to traditional DP accountants, which are used to track the expenditure of…
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes
Marten van Dijk, Nhuong V. Nguyen, Toan N. Nguyen +3
Hogwild! implements asynchronous Stochastic Gradient Descent (SGD) where multiple threads in parallel access a common repository containing training data, perform SGD iterations an…