Toward Model Parallelism for Deep Neural Network based on Gradient-free ADMM Framework

Junxiang Wang, Zheng Chai, Yue Cheng +1

Alternating Direction Method of Multipliers (ADMM) has recently been proposed as a potential alternative optimizer to the Stochastic Gradient Descent(SGD) for deep learning problem…