Randomized Matrix Sketching for Neural Network Training and Gradient Monitoring

Harbir Antil, Deepanshu Verma

Neural network training relies on gradient computation through backpropagation, yet memory requirements for storing layer activations present significant scalability challenges. We…