Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platform

Chi-Chung Chen, Chia-Lin Yang, Hsiang-Yun Cheng

The training process of Deep Neural Network (DNN) is compute-intensive, often taking days to weeks to train a DNN model. Therefore, parallel execution of DNN training on GPUs is a…