Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks

Kevin Lee Hunter, Lawrence Spracklen, Subutai Ahmad

In principle, sparse neural networks should be significantly more efficient than traditional dense networks. Neurons in the brain exhibit two types of sparsity; they are sparsely i…