11 citations · 37 across the 19 of their papers we have counts for
6 papers · 1 filter
Momentum Benefits Non-IID Federated Learning Simply and Provably
Ziheng Cheng, Xinmeng Huang, Pengfei Wu +1
Federated learning is a powerful paradigm for large-scale machine learning, but it faces significant challenges due to unreliable network connections, slow communication, and subst…
Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
Yutong He, Xinmeng Huang, Kun Yuan
Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. How…
Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization
Kun Yuan, Xinmeng Huang, Yiming Chen +3
Decentralized optimization is effective to save communication in large-scale machine learning. Although numerous algorithms have been proposed with theoretical guarantees and empir…
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression
Xinmeng Huang, Yiming Chen, Wotao Yin +1
Recent advances in distributed optimization and learning have shown that communication compression is one of the most effective means of reducing communication. While there have be…
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training
Kun Yuan, Yiming Chen, Xinmeng Huang +4
The scale of deep learning nowadays calls for efficient distributed training algorithms. Decentralized momentum SGD (DmSGD), in which each node averages only with its neighbors, is…
Improved Analysis and Rates for Variance Reduction under Without-replacement Sampling Orders
Xinmeng Huang, Kun Yuan, Xianghui Mao +1
When applying a stochastic algorithm, one must choose an order to draw samples. The practical choices are without-replacement sampling orders, which are empirically faster and more…