Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study

Suyog Gupta, Wei Zhang, Fei Wang

This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochast…