Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition

Abstract

We propose a simple two-step approach for speeding up convolution layers within large convolutional neural networks based on tensor decomposition and discriminative fine-tuning. Given a layer, we use non-linear least squares to compute a low-rank CP-decomposition of the 4D convolution kernel tensor into a sum of a small number of rank-one tensors. At the second step, this decomposition is used to replace the original convolutional layer with a sequence of four convolutional layers with small kernels. After such replacement, the entire network is fine-tuned on the training data using standard backpropagation process. We evaluate this approach on two CNNs and show that it is competitive with previous approaches, leading to higher obtained CPU speedups at the cost of lower accuracy drops for the smaller of the two networks. Thus, for the 36-class character classification CNN, our approach obtains a 8.5x CPU speedup of the whole network with only minor accuracy drop (1% from 91% to 90%). For the standard ImageNet architecture (AlexNet), the approach speeds up the second convolution layer by a factor of 4x at the cost of $1\%$ increase of the overall top-5 classification error.

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Fast Training of Convolutional Networks through FFTs

Cited by in corpus (129)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• Texture Synthesis Using Convolutional Neural Networks
• Convolutional neural networks with low-rank regularization
• Channel Pruning for Accelerating Very Deep Neural Networks
• MCUNet: Tiny Deep Learning on IoT Devices
• Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Recurrent Neural Networks: An Embedded Computing Perspective
• On the Expressive Power of Deep Learning: A Tensor Analysis
• XNOR-Net++: Improved Binary Neural Networks
• Loss-aware Weight Quantization of Deep Networks
• GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving
• Convolutional Tensor-Train LSTM for Spatio-temporal Learning
• What is the State of Neural Network Pruning?
• DyNet: Dynamic Convolution for Accelerating Convolutional Neural Networks
• Towards the Limit of Network Quantization
• Tensor-Train Recurrent Neural Networks for Video Classification
• Tensor Computation: A New Framework for High-Dimensional Problems in EDA
• Mobile Video Object Detection with Temporally-Aware Feature Maps
• MobileDets: Searching for Object Detection Architectures for Mobile Accelerators
• ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks
• Improving Efficiency in Convolutional Neural Network with Multilinear Filters
• Multilinear Compressive Learning
• Compressing Convolutional Neural Networks
• Compressing RNNs for IoT devices by 15-38x using Kronecker Products
• Sharing Residual Units Through Collective Tensor Factorization in Deep Neural Networks
• Exploiting Local Structures with the Kronecker Layer in Convolutional Networks
• Wide Compression: Tensor Ring Nets
• A literature survey of matrix methods for data science
• Learned Threshold Pruning
• A Programmable Approach to Neural Network Compression
• Compression and Interpretability of Deep Neural Networks via Tucker Tensor Layer: From First Principles to Tensor Valued Back-Propagation
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
• Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition
• Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks
• Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution
• Learning Efficient Detector with Semi-supervised Adaptive Distillation
• Knowledge Squeezed Adversarial Network Compression
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks
• Revisiting Dynamic Convolution via Matrix Decomposition
• BT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
• PruneNet: Channel Pruning via Global Importance
• Coordinating Filters for Faster Deep Neural Networks
• Understanding Generalization in Deep Learning via Tensor Methods
• Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
• INsight: A Neuromorphic Computing System for Evaluation of Large Neural Networks
• On-FPGA Training with Ultra Memory Reduction: A Low-Precision Tensor Method
• Rethinking Class-Discrimination Based CNN Channel Pruning
• Privacy-preserving Learning via Deep Net Pruning
• PENNI: Pruned Kernel Sharing for Efficient CNN Inference
• Toward Compact Deep Neural Networks via Energy-Aware Pruning
• Structured Convolutions for Efficient Neural Network Design
• Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks
• The trouble with tensor ring decompositions
• Faster Neural Network Training with Approximate Tensor Operations
• Iterative Low-Rank Approximation for CNN Compression
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• Learning Filter Basis for Convolutional Neural Network Compression
• LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution
• NeCPD: An Online Tensor Decomposition with Optimal Stochastic Gradient Descent
• Learning Efficient Convolutional Networks through Irregular Convolutional Kernels
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification
• Neural Pruning via Growing Regularization
• Kernelized Support Tensor Train Machines
• Compressing CNN Kernels for Videos Using Tucker Decompositions: Towards Lightweight CNN Applications
• Manifold Regularized Dynamic Network Pruning
• Finding trainable sparse networks through Neural Tangent Transfer
• WSNet: Compact and Efficient Networks Through Weight Sampling
• Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
• Principal Component Networks: Parameter Reduction Early in Training
• Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
• Network Automatic Pruning: Start NAP and Take a Nap
• Graph-Adaptive Pruning for Efficient Inference of Convolutional Neural Networks
• Parameter Efficient Deep Neural Networks with Bilinear Projections
• Adaptive Low-Rank Factorization to regularize shallow and deep neural networks
• Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions
• A Support Tensor Train Machine
• TDC: Towards Extremely Efficient CNNs on GPUs via Hardware-Aware Tucker Decomposition
• ThumbNet: One Thumbnail Image Contains All You Need for Recognition
• Patchwork: A Patch-wise Attention Network for Efficient Object Detection and Segmentation in Video Streams
• CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compression
• MobiVSR: A Visual Speech Recognition Solution for Mobile Devices
• Frequency-Weighted Robust Tensor Principal Component Analysis
• Going Beyond Classification Accuracy Metrics in Model Compression
• Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization
• Efficient Semantic Segmentation using Gradual Grouping
• Semi-tensor Product-based TensorDecomposition for Neural Network Compression
• Gator: Customizable Channel Pruning of Neural Networks with Gating
• 3U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration
• PASTA: A Parallel Sparse Tensor Algorithm Benchmark Suite
• Mining the Weights Knowledge for Optimizing Neural Network Structures
• Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization
• Convolutional neural networks compression with low rank and sparse tensor decompositions
• Channel Pruning Guided by Classification Loss and Feature Importance
• A Miniaturized Semantic Segmentation Method for Remote Sensing Image
• Parallelized Tensor Train Learning of Polynomial Classifiers
• Class-dependent Compression of Deep Neural Networks
• Block-term Tensor Neural Networks
• Adaptive Low-Rank Regularization with Damping Sequences to Restrict Lazy Weights in Deep Networks
• Online Tensor-Based Learning for Multi-Way Data
• Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework
• Class-Discriminative CNN Compression
• Deep Model Compression Via Two-Stage Deep Reinforcement Learning
• Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
• Compact Autoregressive Network
• Time-Aware Tensor Decomposition for Missing Entry Prediction
• ViP: Virtual Pooling for Accelerating CNN-based Image Classification and Object Detection
• Faster Convolution Inference Through Using Pre-Calculated Lookup Tables
• Multilinear Map Layer: Prediction Regularization by Structural Constraint
• Binary Neural Network for Speaker Verification
• Low-Rank Embedding of Kernels in Convolutional Neural Networks under Random Shuffling
• Penetrating the Fog: the Path to Efficient CNN Models
• Auto Deep Compression by Reinforcement Learning Based Actor-Critic Structure
• Tucker Tensor Decomposition on FPGA
• clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions
• Magnitude and Uncertainty Pruning Criterion for Neural Networks
• A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs
• An Improving Framework of regularization for Network Compression
• HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression
• Fixed-point Factorized Networks
• A Note on Latency Variability of Deep Neural Networks for Mobile Inference
• DAC: Data-free Automatic Acceleration of Convolutional Networks
• Online Filter Clustering and Pruning for Efficient Convnets