Training DNN Models over Heterogeneous Clusters with Optimal Performance

Chengyi Nie, Jessica Maghakian, Zhenhua Liu

Adjusting batch sizes and adaptively tuning other hyperparameters can significantly speed up deep neural network (DNN) training. Despite the ubiquity of heterogeneous clusters, exi…