Optimal Gradient Checkpointing for Sparse and Recurrent Architectures using Off-Chip Memory

Wadjih Bencheikh, Jan Finkbeiner, Emre Neftci

Recurrent neural networks (RNNs) are valued for their computational efficiency and reduced memory requirements on tasks involving long sequence lengths but require high memory-proc…