575 citations · 1k across the 18 of their papers we have counts for
9 papers · 1 filter
Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
Yae Jee Cho, Jianyu Wang, Gauri Joshi
Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing. Several…
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
Jianyu Wang, Qinghua Liu, Hao Liang +2
In federated optimization, heterogeneity in the clients' local datasets and computation speeds results in large variations in the number of local updates performed by each client i…
Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD
Jianyu Wang, Hao Liang, Gauri Joshi
Distributed stochastic gradient descent (SGD) is essential for scaling the machine learning algorithms to a large number of computing nodes. However, the infrastructures variabilit…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Accelerating Deep Learning by Focusing on the Biggest Losers
Angela H. Jiang, Daniel L. -K. Wong, Giulio Zhou +8
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Select…
MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling
Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang +2
This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network. While a…
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD
Jianyu Wang, Gauri Joshi
Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random c…
Active Distribution Learning from Indirect Samples
Samarth Gupta, Gauri Joshi, Osman Yağan
This paper studies the problem of {\em learning} the probability distribution of a discrete random variable using indirect and sequential samples. At each time step, we c…
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
Jianyu Wang, Gauri Joshi
Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalab…