78 citations · 177 across the 10 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.DC2021★ 40 cited
Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Models
Saeed Rashidi, William Won, Sudarshan Srinivasan +2
Distributed training is a solution to reduce DNN training time by splitting the task across multiple NPUs (e.g., GPU/TPU). However, distributed training adds communication overhead…
cs.DC2021★ 6 cited
LIBRA: Enabling Workload-aware Multi-dimensional Network Topology Optimization for Distributed Training of Large AI Models
William Won, Saeed Rashidi, Sudarshan Srinivasan +1
As model sizes in machine learning continue to scale, distributed training is necessary to accommodate model weights within each device and to reduce training time. However, this c…