Embedding Differentiable Sparsity into Deep Neural Network
arXiv:2006.13716
Abstract
In this paper, we propose embedding sparsity into the structure of deep neural networks, where model parameters can be exactly zero during training with the stochastic gradient descent. Thus, it can learn the sparsified structure and the weights of networks simultaneously. The proposed approach can learn structured as well as unstructured sparsity.
arXiv admin note: text overlap with arXiv:1910.03201