Gradient Under Microscope: Benchmarking Resource Utilization of Memory-Efficient Gradient Computation Methods

Sarthak Mahapatra, Zihan Zhou, Khatoon Khedri +2

AI training's rising resource intensity is straining electricity supplies and carbon budgets, motivating systematic study of memory-efficient training on constrained hardware. We b…