Sparse Training of Neural Networks based on Multilevel Mirror Descent

Yannick Lunk, Sebastian J. Scott, Leon Bungert

We introduce a dynamic sparse training algorithm based on linearized Bregman iterations / mirror descent that exploits the naturally incurred sparsity by alternating between period…