Towards Evolutional Compression

Abstract

Compressing convolutional neural networks (CNNs) is essential for transferring the success of CNNs to a wide variety of applications to mobile devices. In contrast to directly recognizing subtle weights or filters as redundant in a given CNN, this paper presents an evolutionary method to automatically eliminate redundant convolution filters. We represent each compressed network as a binary individual of specific fitness. Then, the population is upgraded at each evolutionary iteration using genetic operations. As a result, an extremely compact CNN is generated using the fittest individual. In this approach, either large or small convolution filters can be redundant, and filters in the compressed network are more distinct. In addition, since the number of filters in each convolutional layer is reduced, the number of filter channels and the size of feature maps are also decreased, naturally improving both the compression and speed-up ratios. Experiments on benchmark deep CNN models suggest the superiority of the proposed algorithm over the state-of-the-art compression methods.

References in corpus (6)

• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Learning Structured Sparsity in Deep Neural Networks
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Cited by in corpus (3)

• Automatically Searching for U-Net Image Translator Architecture
• Searching for Accurate Binary Neural Architectures
• Balanced Binary Neural Networks with Gated Residual