24 citations · 26 across the 5 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 2 cited
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees
Jue Wang, Binhang Yuan, Luka Rimanic +5
Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks. Despite recent intensive…
cs.DC2022★ 24 cited
Decentralized Training of Foundation Models in Heterogeneous Environments
Binhang Yuan, Yongjun He, Jared Quincy Davis +6
Training foundation models, such as GPT-3 and PaLM, can be extremely expensive, often involving tens of thousands of GPUs running continuously for months. These models are typicall…