Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance

XinYu Piao, DoangJoo Synn, JooYoung Park +1

Recent deep learning models are difficult to train using a large batch size, because commodity machines may not have enough memory to accommodate both the model and a large data ba…