An Evaluation of Memory Optimization Methods for Training Neural Networks

Xiaoxuan Liu, Siddharth Jha, Alvin Cheung

As models continue to grow in size, the development of memory optimization methods (MOMs) has emerged as a solution to address the memory bottleneck encountered when training large…