97 citations · 625 across the 24 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.DC2021★ 1 cited
Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters
Jayashree Mohan, Amar Phanishayee, Janardhan Kulkarni +1
Training Deep Neural Networks (DNNs) is a widely popular workload in both enterprises and cloud data centers. Existing schedulers for DNN training consider GPU as the dominant reso…
cs.CL2021★ 10 cited
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
Deepak Narayanan, Mohammad Shoeybi, Jared Casper +9
Large language models have led to state-of-the-art accuracies across a range of tasks. However, training these models efficiently is challenging for two reasons: a) GPU memory capa…