23 citations · 58 across the 11 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.DC2020★ 12 cited
Domain-specific Communication Optimization for Distributed DNN Training
Hao Wang, Jingrong Chen, Xinchen Wan +7
Communication overhead poses an important obstacle to distributed DNN training and draws increasing attention in recent years. Despite continuous efforts, prior solutions such as g…
cs.LG2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training
Weiyan Wang, Cengguang Zhang, Liu Yang +2
Bulk synchronous parallel (BSP) is the de-facto paradigm for distributed DNN training in today's production clusters. However, due to the global synchronization nature, its perform…