Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU

Jafar Badour, Maurice van Keulen, Elena Mocanu

Deep neural networks continue to grow in parameter count, driving up training and inference cost on GPUs. Sparse neural networks and Dynamic Sparse Training (DST) promise to reduce…