Efficient Strong Scaling Through Burst Parallel Training

Seo Jin Park, Joshua Fried, Sunghyun Kim +2

As emerging deep neural network (DNN) models continue to grow in size, using large GPU clusters to train DNNs is becoming an essential requirement to achieving acceptable training…