873 citations · 1.3k across the 35 of their papers we have counts for
6 papers · 1 filter
Optimal Complexity in Non-Convex Decentralized Learning over Time-Varying Networks
Xinmeng Huang, Kun Yuan
Decentralized optimization with time-varying networks is an emerging paradigm in machine learning. It saves remarkable communication overhead in large-scale deep training and is mo…
Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization
Kun Yuan, Xinmeng Huang, Yiming Chen +3
Decentralized optimization is effective to save communication in large-scale machine learning. Although numerous algorithms have been proposed with theoretical guarantees and empir…
On the Performance of Gradient Tracking with Local Updates
Edward Duc Hien Nguyen, Sulaiman A. Alghunaim, Kun Yuan +1
We study the decentralized optimization problem where a network of agents seeks to minimize the average of a set of heterogeneous non-convex cost functions distributedly. State…
Communication-Efficient Topologies for Decentralized Learning with Consensus Rate
Zhuoqing Song, Weijian Li, Kexin Jin +4
Decentralized optimization is an emerging paradigm in distributed learning in which agents achieve network-wide solutions by peer-to-peer communication without the central server.…
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression
Xinmeng Huang, Yiming Chen, Wotao Yin +1
Recent advances in distributed optimization and learning have shown that communication compression is one of the most effective means of reducing communication. While there have be…
Heavy-Tail Phenomenon in Decentralized SGD
Mert Gurbuzbalaban, Yuanhan Hu, Umut Simsekli +2
Recent theoretical studies have shown that heavy-tails can emerge in stochastic optimization due to `multiplicative noise', even under surprisingly simple settings, such as linear…