Weighted parallel SGD for distributed unbalanced-workload training system

Cheng Daning, Li Shigang, Zhang Yunquan

Stochastic gradient descent (SGD) is a popular stochastic optimization method in machine learning. Traditional parallel SGD algorithms, e.g., SimuParallel SGD, often require all no…