DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation

Yu Tang, Chenyu Wang, Yufan Zhang +5

The further development of deep neural networks is hampered by the limited GPU memory resource. Therefore, the optimization of GPU memory resources is highly demanded. Swapping and…