125 citations · 409 across the 32 of their papers we have counts for
5 papers · 1 filter
ErrorCompensatedX: error compensation for variance reduced algorithms
Hanlin Tang, Yao Li, Ji Liu +1
Communication cost is one major bottleneck for the scalability for distributed learning. One approach to reduce the communication cost is to compress the gradient during communicat…
APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm
Hanlin Tang, Shaoduo Gan, Samyam Rajbhandari +4
Adam is the important optimization algorithm to guarantee efficiency and accuracy for training many important tasks such as BERT and ImageNet. However, Adam is generally not compat…
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
Hanlin Tang, Xiangru Lian, Chen Yu +2
A standard approach in large scale machine learning is distributed stochastic gradient training, which requires the computation of aggregated stochastic gradients over multiple nod…
Distributed Learning over Unreliable Networks
Chen Yu, Hanlin Tang, Cedric Renggli +5
Most of today's distributed machine learning systems assume {\em reliable networks}: whenever two machines exchange information (e.g., gradients or models), the network should guar…
D: Decentralized Training over Decentralized Data
Hanlin Tang, Xiangru Lian, Ming Yan +2
While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from differ…