Slim-DP: A Light Communication Data Parallelism for DNN

Shizhao Sun, Wei Chen, Jiang Bian +2

Data parallelism has emerged as a necessary technique to accelerate the training of deep neural networks (DNN). In a typical data parallelism approach, the local workers push the l…