SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks

Mahdi Nikdan, Tommaso Pegolotti, Eugenia Iofinova +2

We provide a new efficient version of the backpropagation algorithm, specialized to the case where the weights of the neural network being trained are sparse. Our algorithm is gene…