Stanza: Layer Separation for Distributed Training in Deep Learning

Xiaorui Wu, Hong Xu, Bo Li +1

The parameter server architecture is prevalently used for distributed deep learning. Each worker machine in a parameter server system trains the complete model, which leads to a he…