Improving Efficiency in Convolutional Neural Network with Multilinear Filters

Abstract

The excellent performance of deep neural networks has enabled us to solve several automatization problems, opening an era of autonomous devices. However, current deep net architectures are heavy with millions of parameters and require billions of floating point operations. Several works have been developed to compress a pre-trained deep network to reduce memory footprint and, possibly, computation. Instead of compressing a pre-trained network, in this work, we propose a generic neural network layer structure employing multilinear projection as the primary feature extractor. The proposed architecture requires several times less memory as compared to the traditional Convolutional Neural Networks (CNN), while inherits the similar design principles of a CNN. In addition, the proposed architecture is equipped with two computation schemes that enable computation reduction or scalability. Experimental results show the effectiveness of our compact projection that outperforms traditional CNN, while requiring far fewer parameters.

10 pages, 3 figures

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Striving for Simplicity: The All Convolutional Net
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Compressing Deep Convolutional Networks using Vector Quantization
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Learning Structured Sparsity in Deep Neural Networks
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Dynamic Network Surgery for Efficient DNNs
• Compressing Convolutional Neural Networks
• Design of Efficient Convolutional Layers using Single Intra-channel Convolution, Topological Subdivisioning and Spatial "Bottleneck" Structure
• Tensor Contraction Layers for Parsimonious Deep Nets

Cited by in corpus (4)

• Heterogeneous Multilayer Generalized Operational Perceptron
• Multilinear Compressive Learning
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Knowledge Distillation By Sparse Representation Matching