Tunable Subnetwork Splitting for Model-parallelism of Neural Network Training

Junxiang Wang, Zheng Chai, Yue Cheng +1

Alternating minimization methods have recently been proposed as alternatives to the gradient descent for deep neural network optimization. Alternating minimization methods can typi…