Speeding up Convolutional Neural Networks with Low Rank Expansions

Abstract

The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability. Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain. Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition, showing a possible 2.5x speedup with no loss in accuracy, and 4.5x speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.

References in corpus (6)

• Improving neural networks by preventing co-adaptation of feature detectors
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Predicting Parameters in Deep Learning
• DenseNet: Implementing Efficient ConvNet Descriptor Pyramids
• Fast Training of Convolutional Networks through FFTs
• End-to-End Text Recognition with Hybrid HMM Maxout Models

Cited by in corpus (255)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• LoRA: Low-Rank Adaptation of Large Language Models
• FitNets: Hints for Thin Deep Nets
• Compressing Deep Convolutional Networks using Vector Quantization
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Pruning Filters for Efficient ConvNets
• Learning Structured Sparsity in Deep Neural Networks
• Texture Synthesis Using Convolutional Neural Networks
• Aggregated Residual Transformations for Deep Neural Networks
• Convolutional neural networks with low-rank regularization
• Channel Pruning for Accelerating Very Deep Neural Networks
• Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Recent Advances in Convolutional Neural Networks
• Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
• Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
• Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• Scene Text Detection via Holistic, Multi-Channel Prediction
• Scalable Methods for 8-bit Training of Neural Networks
• Prospects and applications of photonic neural networks
• Discrimination-aware Channel Pruning for Deep Neural Networks
• Image Super-Resolution Using Deep Convolutional Networks
• Exploring Sparsity in Recurrent Neural Networks
• Faster CNNs with Direct Sparse Convolutions and Guided Pruning
• Compression-aware Training of Deep Networks
• Memory Bounded Deep Convolutional Networks
• Edge Intelligence: Architectures, Challenges, and Applications
• Compressing Neural Networks using the Variational Information Bottleneck
• Learning Intrinsic Sparse Structures within Long Short-Term Memory
• Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
• Light-Weight RefineNet for Real-Time Semantic Segmentation
• Sequence-Level Knowledge Distillation
• From Facial Parts Responses to Face Detection: A Deep Learning Approach
• Focus: Querying Large Video Datasets with Low Latency and Low Cost
• The Power of Sparsity in Convolutional Neural Networks
• GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving
• ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks
• Interleaved Group Convolutions for Deep Neural Networks
• Network Decoupling: From Regular to Depthwise Separable Convolutions
• A Low Effort Approach to Structured CNN Design Using PCA
• DyNet: Dynamic Convolution for Accelerating Convolutional Neural Networks
• Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Effective Quantization Methods for Recurrent Neural Networks
• NISP: Pruning Networks using Neuron Importance Score Propagation
• Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
• Compacting Deep Neural Networks for Internet of Things: Methods and Applications
• A Survey on Deep Learning Methods for Robot Vision
• Batch-Shaping for Learning Conditional Channel Gated Networks
• MEC: Memory-efficient Convolution for Deep Neural Network
• Tensorized Embedding Layers for Efficient Model Compression
• An exploration of parameter redundancy in deep networks with circulant projections
• Mobile Video Object Detection with Temporally-Aware Feature Maps
• Improving Efficiency in Convolutional Neural Network with Multilinear Filters
• More is Less: A More Complicated Network with Less Inference Complexity
• Multilinear Compressive Learning
• DecomposeMe: Simplifying ConvNets for End-to-End Learning
• AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference
• SCSP: Spectral Clustering Filter Pruning with Soft Self-adaption Manners
• Compressing Convolutional Neural Networks
• Quantized Convolutional Neural Networks for Mobile Devices
• Accelerating Very Deep Convolutional Networks for Classification and Detection
• Sharing Residual Units Through Collective Tensor Factorization in Deep Neural Networks
• Exploiting Local Structures with the Kronecker Layer in Convolutional Networks
• Convolutional Neural Networks at Constrained Time Cost
• IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
• Learning to Prune Filters in Convolutional Neural Networks
• Model compression as constrained optimization, with application to neural nets. Part I: general framework
• Communication-Efficient Edge AI: Algorithms and Systems
• Task dependent Deep LDA pruning of neural networks
• Knowledge Distillation Methods for Efficient Unsupervised Adaptation Across Multiple Domains
• Convolution by Evolution: Differentiable Pattern Producing Networks
• A Programmable Approach to Neural Network Compression
• Performance Guaranteed Network Acceleration via High-Order Residual Quantization
• Parallel Separable 3D Convolution for Video and Volumetric Data Understanding
• Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank
• SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
• Binarized Neural Networks on the ImageNet Classification Task
• Fast ConvNets Using Group-wise Brain Damage
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
• Evolutionary Synthesis of Deep Neural Networks via Synaptic Cluster-driven Genetic Encoding
• Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition
• Deep Fried Convnets
• Selfish Sparse RNN Training
• Model Compression with Multi-Task Knowledge Distillation for Web-scale Question Answering System
• Knowledge Projection for Deep Neural Networks
• SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
• Grow and Prune Compact, Fast, and Accurate LSTMs
• Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks
• Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
• EffNet: An Efficient Structure for Convolutional Neural Networks
• FD-MobileNet: Improved MobileNet with a Fast Downsampling Strategy
• Rethinking Bottleneck Structure for Efficient Mobile Network Design
• Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale
• Zero-Shot Knowledge Distillation from a Decision-Based Black-Box Model
• SBNet: Sparse Blocks Network for Fast Inference
• Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks
• Knowledge Squeezed Adversarial Network Compression
• Trained Rank Pruning for Efficient Deep Neural Networks
• Functionality-Oriented Convolutional Filter Pruning
• Search for Better Students to Learn Distilled Knowledge
• Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
• LCNN: Lookup-based Convolutional Neural Network
• Fix your classifier: the marginal value of training the last weight layer
• BT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
• Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions
• Grid Loss: Detecting Occluded Faces
• RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks
• Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networks
• Understanding Generalization in Deep Learning via Tensor Methods
• PruneNet: Channel Pruning via Global Importance
• Coordinating Filters for Faster Deep Neural Networks
• Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
• Compressing Neural Language Models by Sparse Word Representations
• Rethinking Class-Discrimination Based CNN Channel Pruning
• XSepConv: Extremely Separated Convolution
• Hard-Attention for Scalable Image Classification
• CUP: Cluster Pruning for Compressing Deep Neural Networks
• GASL: Guided Attention for Sparsity Learning in Deep Neural Networks
• Dynamic Capacity Networks
• Toward Compact Deep Neural Networks via Energy-Aware Pruning
• Structured Convolutions for Efficient Neural Network Design
• Learning Versatile Convolution Filters for Efficient Visual Recognition
• Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks
• Constrained deep neural network architecture search for IoT devices accounting hardware calibration
• Multiscale Hierarchical Convolutional Networks
• Exploring Gradient Flow Based Saliency for DNN Model Compression
• Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System
• Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks
• Improving Neural Network Training in Low Dimensional Random Bases
• Convolutional Neural Network Simplification with Progressive Retraining
• Neural Epitome Search for Architecture-Agnostic Network Compression
• Harmonic Networks: Integrating Spectral Information into CNNs
• Demystifying Neural Network Filter Pruning
• Sparsifying Neural Network Connections for Face Recognition
• Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds
• Pufferfish: Communication-efficient Models At No Extra Cost
• Compressing complex convolutional neural network based on an improved deep compression algorithm
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution
• Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification
• Data Efficient Stagewise Knowledge Distillation
• SparCE: Sparsity aware General Purpose Core Extensions to Accelerate Deep Neural Networks
• Learning Architectures for Binary Networks
• PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution
• DelugeNets: Deep Networks with Efficient and Flexible Cross-layer Information Inflows
• GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Dynamic Group Convolution for Accelerating Convolutional Neural Networks
• REPrune: Filter Pruning via Representative Election
• Finding trainable sparse networks through Neural Tangent Transfer
• BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization
• Compression of Fully-Connected Layer in Neural Network by Kronecker Product
• Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration
• Automatic Rank Selection for High-Speed Convolutional Neural Network
• NeRV: Neural Representations for Videos
• WSNet: Compact and Efficient Networks Through Weight Sampling
• Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
• Network Automatic Pruning: Start NAP and Take a Nap
• Compressing LSTM Networks by Matrix Product Operators
• Compression of Deep Neural Networks on the Fly
• Joint Architecture and Knowledge Distillation in CNN for Chinese Text Recognition
• Efficient Neural Network Compression
• HGC: Hierarchical Group Convolution for Highly Efficient Neural Network
• Correlation Congruence for Knowledge Distillation
• Development of deep learning algorithms to categorize free-text notes pertaining to diabetes: convolution neural networks achieve higher accuracy than support vector machines
• SECS: Efficient Deep Stream Processing via Class Skew Dichotomy
• Knowledge Distillation via Instance-level Sequence Learning
• Optimal Quantization for Batch Normalization in Neural Network Deployments and Beyond
• Computation-Performance Optimization of Convolutional Neural Networks with Redundant Kernel Removal
• Principal Component Networks: Parameter Reduction Early in Training
• Compact representations of convolutional neural networks via weight pruning and quantization
• EasyConvPooling: Random Pooling with Easy Convolution for Accelerating Training and Testing
• Adaptive Low-Rank Factorization to regularize shallow and deep neural networks
• Merging and Evolution: Improving Convolutional Neural Networks for Mobile Applications
• Parameter Efficient Deep Neural Networks with Bilinear Projections
• MUSCO: Multi-Stage Compression of neural networks
• Deep Neural Network Approximation using Tensor Sketching
• BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements
• A Feature-map Discriminant Perspective for Pruning Deep Neural Networks
• Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization
• DeepThin: A Self-Compressing Library for Deep Neural Networks
• ThumbNet: One Thumbnail Image Contains All You Need for Recognition
• Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence
• Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization
• BasisConv: A method for compressed representation and learning in CNNs
• Truncating Wide Networks using Binary Tree Architectures
• Recent Advances in Convolutional Neural Network Acceleration
• Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks
• Fast Training of Convolutional Neural Networks via Kernel Rescaling
• Towards Deep Compositional Networks
• Efficient Semantic Segmentation using Gradual Grouping
• Deep Learning Acceleration Techniques for Real Time Mobile Vision Applications
• Class-dependent Compression of Deep Neural Networks
• Full-Stack Filters to Build Minimum Viable CNNs
• Improving Network Slimming with Nonconvex Regularization
• Efficient and Robust Machine Learning for Real-World Systems
• Composite Binary Decomposition Networks
• A Mixture of Expert Approach for Low-Cost Customization of Deep Neural Networks
• Semi-tensor Product-based TensorDecomposition for Neural Network Compression
• Mining the Weights Knowledge for Optimizing Neural Network Structures
• PBGen: Partial Binarization of Deconvolution-Based Generators for Edge Intelligence
• Artificial Intelligence for 5G Wireless Systems: Opportunities, Challenges, and Future Research Directions
• E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAs
• Studying the Plasticity in Deep Convolutional Neural Networks using Random Pruning
• CircConv: A Structured Convolution with Low Complexity
• Building Fast and Compact Convolutional Neural Networks for Offline Handwritten Chinese Character Recognition
• Recurrent Residual Module for Fast Inference in Videos
• Class-Discriminative CNN Compression
• Approximate Random Dropout
• Architecture Aware Latency Constrained Sparse Neural Networks
• Deep Learning Approximation: Zero-Shot Neural Network Speedup
• Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework
• Auto Deep Compression by Reinforcement Learning Based Actor-Critic Structure
• Dep-$L_0$: Improving $L_0$-based Network Sparsification via Dependency Modeling
• Multi-objective Evolutionary Approach for Efficient Kernel Size and Shape for CNN
• Binarized Convolutional Neural Networks for Efficient Inference on GPUs
• FastSal: a Computationally Efficient Network for Visual Saliency Prediction
• Rank-1 Convolutional Neural Network
• Softer Pruning, Incremental Regularization
• Adaptive Low-Rank Regularization with Damping Sequences to Restrict Lazy Weights in Deep Networks
• Fixed-point Factorized Networks
• Compression-aware Continual Learning using Singular Value Decomposition
• Efficient Fusion of Sparse and Complementary Convolutions
• Radius Adaptive Convolutional Neural Network
• New Results on CGR/CUR Approximation of a Matrix
• Magnitude and Uncertainty Pruning Criterion for Neural Networks
• Activation Map Adaptation for Effective Knowledge Distillation
• Block-term Tensor Neural Networks
• DAC: Data-free Automatic Acceleration of Convolutional Networks
• Grow-Push-Prune: aligning deep discriminants for effective structural network compression
• A Highly Effective Low-Rank Compression of Deep Neural Networks with Modified Beam-Search and Modified Stable Rank
• Deeply Shared Filter Bases for Parameter-Efficient Convolutional Neural Networks
• FC2T2: The Fast Continuous Convolutional Taylor Transform with Applications in Vision and Graphics
• Multi-Precision Quantized Neural Networks via Encoding Decomposition of -1 and +1
• Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting
• The Efficiency Misnomer
• Training compact deep learning models for video classification using circulant matrices
• ADA-Tucker: Compressing Deep Neural Networks via Adaptive Dimension Adjustment Tucker Decomposition
• CNN inference acceleration using dictionary of centroids
• Efficient Semantic Segmentation for Visual Bird's-eye View Interpretation
• Multilinear Map Layer: Prediction Regularization by Structural Constraint
• Efficient Classification of Very Large Images with Tiny Objects
• Reliable Identification of Redundant Kernels for Convolutional Neural Network Compression
• Prune the Convolutional Neural Networks with Sparse Shrink
• clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions
• Knowledge Distillation By Sparse Representation Matching
• Balancing Accuracy and Latency in Multipath Neural Networks
• Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
• PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices