Parallax: Sparsity-aware Data Parallel Training of Deep Neural Networks

Soojeong Kim, Gyeong-In Yu, Hojin Park +6

The employment of high-performance servers and GPU accelerators for training deep neural network models have greatly accelerated recent advances in deep learning (DL). DL framework…