Dynamic Sparse Training with Structured Sparsity

Mike Lasby, Anna Golubeva, Utku Evci +2

Dynamic Sparse Training (DST) methods achieve state-of-the-art results in sparse neural network training, matching the generalization of dense models while enabling sparse training…