GPU Memory Usage Optimization for Backward Propagation in Deep Network Training

Ding-Yong Hong, Tzu-Hsien Tsai, Ning Wang +2

In modern Deep Learning, it has been a trend to design larger Deep Neural Networks (DNNs) for the execution of more complex tasks and better accuracy. On the other hand, Convolutio…