Less Memory Means smaller GPUs: Backpropagation with Compressed Activations

Daniel Barley, Holger Fröning

The ever-growing scale of deep neural networks (DNNs) has lead to an equally rapid growth in computational resource requirements. Many recent architectures, most prominently Large…