Compressing 3DCNNs Based on Tensor Train Decomposition

Abstract

Three dimensional convolutional neural networks (3DCNNs) have been applied in many tasks, e.g., video and 3D point cloud recognition. However, due to the higher dimension of convolutional kernels, the space complexity of 3DCNNs is generally larger than that of traditional two dimensional convolutional neural networks (2DCNNs). To miniaturize 3DCNNs for the deployment in confining environments such as embedded devices, neural network compression is a promising approach. In this work, we adopt the tensor train (TT) decomposition, a straightforward and simple in situ training compression method, to shrink the 3DCNN models. Through proposing tensorizing 3D convolutional kernels in TT format, we investigate how to select appropriate TT ranks for achieving higher compression ratio. We have also discussed the redundancy of 3D convolutional kernels for compression, core significance and future directions of this work, as well as the theoretical computation complexity versus practical executing time of convolution in TT. In the light of multiple contrast experiments based on VIVA challenge, UCF11, and UCF101 datasets, we conclude that TT decomposition can compress 3DCNNs by around one hundred times without significant accuracy loss, which will enable its applications in extensive real world scenarios.

arXiv commentAccepted by Neural Networks. Please see the final version by the DOI below

References in corpus (17)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• Predicting Parameters in Deep Learning
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Tensorizing Neural Networks
• Deformable Convolutional Networks
• Efficient tensor completion for color image and video recovery: Low-rank tensor train
• Action Recognition using Visual Attention
• Training and Inference with Integers in Deep Neural Networks
• Alternating minimal energy methods for linear systems in higher dimensions. Part II: Faster algorithm and application to nonsymmetric systems
• Alternating minimal energy methods for linear systems in higher dimensions. Part I: SPD systems
• Compressing Recurrent Neural Network with Tensor Train
• Ultimate tensorization: compressing convolutional and FC layers alike
• Tensor-Train Recurrent Neural Networks for Video Classification
• Tensor Train decomposition on TensorFlow (T3F)
• Regularized Computation of Approximate Pseudoinverse of Large Matrices Using Low-Rank Tensor Train Decompositions
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression

Cited by in corpus (4)

• Hybrid Tensor Decomposition in Neural Network Compression
• Kronecker CP Decomposition with Fast Multiplication for Compressing RNNs
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions