Structural Compression of Convolutional Neural Networks

Abstract

Deep convolutional neural networks (CNNs) have been successful in many tasks in machine vision, however, millions of weights in the form of thousands of convolutional filters in CNNs makes them difficult for human intepretation or understanding in science. In this article, we introduce CAR, a greedy structural compression scheme to obtain smaller and more interpretable CNNs, while achieving close to original accuracy. The compression is based on pruning filters with the least contribution to the classification accuracy. We demonstrate the interpretability of CAR-compressed CNNs by showing that our algorithm prunes filters with visually redundant functionalities such as color filters. These compressed networks are easier to interpret because they retain the filter diversity of uncompressed networks with order of magnitude less filters. Finally, a variant of CAR is introduced to quantify the importance of each image category to each CNN filter. Specifically, the most and the least important class labels are shown to be meaningful interpretations of each filter.

References in corpus (10)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Pruning Filters for Efficient ConvNets
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Bird Species Categorization Using Pose Normalized Deep Convolutional Nets
• Learning the Number of Neurons in Deep Networks
• Interpreting Convolutional Neural Networks Through Compression

Cited by in corpus (9)

• Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
• Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions
• Brain-Computer Interface in Virtual Reality
• Interpreting Convolutional Neural Networks Through Compression
• Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
• Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning
• Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
• Multi-Stage Prototype Learning for Interpretable Time Series Classification
• Individualized Multilayer Tensor Learning with An Application in Imaging Analysis