CPOT: Channel Pruning via Optimal Transport
arXiv:2005.10451
Abstract
Recent advances in deep neural networks (DNNs) lead to tremendously growing network parameters, making the deployments of DNNs on platforms with limited resources extremely difficult. Therefore, various pruning methods have been developed to compress the deep network architectures and accelerate the inference process. Most of the existing channel pruning methods discard the less important filters according to well-designed filter ranking criteria. However, due to the limited interpretability of deep learning models, designing an appropriate ranking criterion to distinguish redundant filters is difficult. To address such a challenging issue, we propose a new technique of Channel Pruning via Optimal Transport, dubbed CPOT. Specifically, we locate the Wasserstein barycenter for channels of each layer in the deep models, which is the mean of a set of probability distributions under the optimal transport metric. Then, we prune the redundant information located by Wasserstein barycenters. At last, we empirically demonstrate that, for classification tasks, CPOT outperforms the state-of-the-art methods on pruning ResNet-20, ResNet-32, ResNet-56, and ResNet-110. Furthermore, we show that the proposed CPOT technique is good at compressing the StarGAN models by pruning in the more difficult case of image-to-image translation tasks.
arXiv comment11 pages
References in corpus (5)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Fast Computation of Wasserstein Barycenters
- Global Sparse Momentum SGD for Pruning Very Deep Neural Networks
- A Fast Proximal Point Method for Computing Exact Wasserstein Distance