9 papers · 1 filter
Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
Samuel Erickson, Mikael Johansson
In modern machine learning, parallelization of training is an important strategy for increasing scale. Asynchronous stochastic gradient descent (ASGD), which maximizes the utilizat…
Decentralized SGD with Controlled Disagreement Finds Flatter Minima
Zesen Wang, Mikael Johansson
Decentralized training is often regarded as inferior to centralized training because the consensus errors between workers are thought to undermine convergence and generalization. T…
Byzantine-Robust Federated Learning with Learnable Aggregation Weights
Javad Parsa, Amir Hossein Daghestani, André M. H. Teixeira +1
Federated Learning (FL) enables clients to collaboratively train a global model without sharing their private data. However, the presence of malicious (Byzantine) clients poses sig…
Data-Driven Laplacian-Constrained Personalized Federated Learning
Samuel Erickson, Mikael Johansson
Personalization addresses one of the defining challenges in federated learning: statistical heterogeneity among clients. We show that the dissimilarity between the ideal personaliz…
Non-convex composite federated learning with heterogeneous data
Jiaojiao Zhang, Jiang Hu, Mikael Johansson
We propose an innovative algorithm for non-convex composite federated learning that decouples the proximal operator evaluation and the communication between server and clients. Mor…
Locally Differentially Private Online Federated Learning With Correlated Noise
Jiaojiao Zhang, Linglingzhi Zhu, Dominik Fay +1
We introduce a locally differentially private (LDP) algorithm for online federated learning that employs temporally correlated noise to improve utility while preserving privacy. To…
Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data
Jiaojiao Zhang, Jiang Hu, Anthony Man-Cho So +1
Many machine learning tasks, such as principal component analysis and low-rank matrix completion, give rise to manifold optimization problems. Although there is a large body of wor…
Differentially Private Online Federated Learning with Correlated Noise
Jiaojiao Zhang, Linglingzhi Zhu, Mikael Johansson
We introduce a novel differentially private algorithm for online federated learning that employs temporally correlated noise to enhance utility while ensuring privacy of continuous…
Composite federated learning with heterogeneous data
Jiaojiao Zhang, Jiang Hu, Mikael Johansson
We propose a novel algorithm for solving the composite Federated Learning (FL) problem. This algorithm manages non-smooth regularization by strategically decoupling the proximal op…