On the Acceleration of Deep Learning Model Parallelism with Staleness

An Xu, Zhouyuan Huo, Heng Huang

Training the deep convolutional neural network for computer vision problems is slow and inefficient, especially when it is large and distributed across multiple devices. The ineffi…