10 citations · 13 across the 6 of their papers we have counts for
11 papers
PROFIT: A Specialized Optimizer for Deep Fine Tuning
Anirudh S Chakravarthy, Shuai Kyle Zheng, Xin Huang +4
The fine-tuning of pre-trained models has become ubiquitous in generative AI, computer vision, and robotics. Although much attention has been paid to improving the efficiency of fi…
Contractive error feedback for gradient compression
Bingcong Li, Shuai Zheng, Parameswaran Raman +2
On-device memory concerns in distributed deep learning have become severe due to (i) the growth of model size in multi-GPU training, and (ii) the wide adoption of deep neural netwo…
Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence
Shu Zheng, Tiandi Ye, Xiang Li +1
Federated learning (FL) on heterogeneous data (non-IID data) has recently received great attention. Most existing methods focus on studying the convergence guarantees for the globa…
Compressed Communication for Distributed Training: Adaptive Methods and System
Yuchen Zhong, Cong Xie, Shuai Zheng +1
Communication overhead severely hinders the scalability of distributed machine learning systems. Recently, there has been a growing interest in using gradient compression to reduce…
LID 2020: The Learning from Imperfect Data Challenge Results
Yunchao Wei, Shuai Zheng, Ming-Ming Cheng +32
Learning from imperfect data becomes an issue in many industrial applications after the research community has made profound progress in supervised learning from perfectly annotate…
CSER: Communication-efficient SGD with Error Reset
Cong Xie, Shuai Zheng, Oluwasanmi Koyejo +3
The scalability of Distributed Stochastic Gradient Descent (SGD) is today limited by communication bottlenecks. We propose a novel SGD variant: Communication-efficient SGD with Err…