438 citations · 1k across the 57 of their papers we have counts for
6 papers · 1 filter
Dynamic Privacy Allocation for Locally Differentially Private Federated Learning with Composite Objectives
Jiaojiao Zhang, Dominik Fay, Mikael Johansson
This paper proposes a locally differentially private federated learning algorithm for strongly convex but possibly nonsmooth problems that protects the gradients of each worker aga…
Delay-adaptive step-sizes for asynchronous learning
Xuyang Wu, Sindri Magnusson, Hamid Reza Feyzmahdavian +1
In scalable machine learning systems, model training is often parallelized over multiple nodes that run without tight synchronization. Most analysis results for the related asynchr…
On Uniform Boundedness Properties of SGD and its Momentum Variants
Xiaoyu Wang, Mikael Johansson
A theoretical, and potentially also practical, problem with stochastic gradient descent is that trajectories may escape to infinity. In this note, we investigate uniform boundednes…
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Xiaoyu Wang, Mikael Johansson
Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima. We derive…
Advances in Asynchronous Parallel and Distributed Optimization
Mahmoud Assran, Arda Aytekin, Hamid Feyzmahdavian +2
Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed op…
The Convergence of Sparsified Gradient Methods
Dan Alistarh, Torsten Hoefler, Mikael Johansson +3
Distributed training of massive machine learning models, in particular deep neural networks, via Stochastic Gradient Descent (SGD) is becoming commonplace. Several families of comm…