Elastic deep learning in multi-tenant GPU cluster

Yidi Wu, Kaihao Ma, Xiao Yan +6

We study how to support elasticity, i.e., the ability to dynamically adjust the parallelism (number of GPUs), for deep neural network (DNN) training. Elasticity can benefit multi-t…