MalleTrain: Deep Neural Network Training on Unfillable Supercomputer Nodes

Xiaolong Ma, Feng Yan, Lei Yang +4

First-come first-serve scheduling can result in substantial (up to 10%) of transiently idle nodes on supercomputers. Recognizing that such unfilled nodes are well-suited for deep n…