97 citations · 625 across the 24 of their papers we have counts for
5 papers · 1 filter
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads
Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka +2
Specialized accelerators such as GPUs, TPUs, FPGAs, and custom ASICs have been increasingly deployed to train deep learning models. These accelerators exhibit heterogeneous perform…
Analyzing and Mitigating Data Stalls in DNN Training
Jayashree Mohan, Amar Phanishayee, Ashish Raniwala +1
Training Deep Neural Networks (DNNs) is resource-intensive and time-consuming. While prior research has explored many different ways of reducing DNN training time, the impact of in…
Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
Hongyu Zhu, Amar Phanishayee, Gennady Pekhimenko
Modern deep neural network (DNN) training jobs use complex and heterogeneous software/hardware stacks. The efficacy of software-level optimizations can vary significantly when used…
Efficient Algorithms for Device Placement of DNN Graph Operators
Jakub Tarnawski, Amar Phanishayee, Nikhil R. Devanur +2
Modern machine learning workloads use large models, with complex structures, that are very expensive to execute. The devices that execute complex models are becoming increasingly h…
Memory-Efficient Pipeline-Parallel DNN Training
Deepak Narayanan, Amar Phanishayee, Kaiyu Shi +2
Many state-of-the-art ML results have been obtained by scaling up the number of parameters in existing models. However, parameters and activations for such large models often do no…