575 citations · 1.1k across the 10 of their papers we have counts for
11 papers
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Local Adaptivity in Federated Learning: Convergence and Consistency
Jianyu Wang, Zheng Xu, Zachary Garrett +3
The federated learning (FL) framework trains a machine learning model using decentralized data stored at edge client devices by periodically aggregating locally trained models. Pop…
Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
Yae Jee Cho, Jianyu Wang, Gauri Joshi
Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing. Several…
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
Jianyu Wang, Qinghua Liu, Hao Liang +2
In federated optimization, heterogeneity in the clients' local datasets and computation speeds results in large variations in the number of local updates performed by each client i…
Machine Learning on Volatile Instances
Xiaoxi Zhang, Jianyu Wang, Gauri Joshi +1
Due to the massive size of the neural network models and training datasets used in machine learning today, it is imperative to distribute stochastic gradient descent (SGD) by split…
Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD
Jianyu Wang, Hao Liang, Gauri Joshi
Distributed stochastic gradient descent (SGD) is essential for scaling the machine learning algorithms to a large number of computing nodes. However, the infrastructures variabilit…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
Jianyu Wang, Vinayak Tantia, Nicolas Ballas +1
Distributed optimization is essential for training large models on large datasets. Multiple approaches have been proposed to reduce the communication overhead in distributed traini…
MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling
Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang +2
This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network. While a…
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD
Jianyu Wang, Gauri Joshi
Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random c…
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
Jianyu Wang, Gauri Joshi
Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalab…