Showing cs.LGShow all
2 papers · 1 filter
cs.LG2020
PSO-PS: Parameter Synchronization with Particle Swarm Optimization for Distributed Training of Deep Neural Networks
Qing Ye, Yuxuan Han, Yanan sun +1
Parameter updating is an important stage in parallelism-based distributed deep learning. Synchronous methods are widely used in distributed training the Deep Neural Networks (DNNs)…
cs.LG2020
HPSGD: Hierarchical Parallel SGD With Stale Gradients Featuring
Yuhao Zhou, Qing Ye, Hailun Zhang +1
While distributed training significantly speeds up the training process of the deep neural network (DNN), the utilization of the cluster is relatively low due to the time-consuming…