MPDCompress - Matrix Permutation Decomposition Algorithm for Deep Neural Network Compression
arXiv:1805.12085
Abstract
Deep neural networks (DNNs) have become the state-of-the-art technique for machine learning tasks in various applications. However, due to their size and the computational complexity, large DNNs are not readily deployable on edge devices in real-time. To manage complexity and accelerate computation, network compression techniques based on pruning and quantization have been proposed and shown to be effective in reducing network size. However, such network compression can result in irregular matrix structures that are mismatched with modern hardware-accelerated platforms, such as graphics processing units (GPUs) designed to perform the DNN matrix multiplications in a structured (block-based) way. We propose MPDCompress, a DNN compression algorithm based on matrix permutation decomposition via random mask generation. In-training application of the masks molds the synaptic weight connection matrix to a sub-graph separation format. Aided by the random permutations, a hardware-desirable block matrix is generated, allowing for a more efficient implementation and compression of the network. To show versatility, we empirically verify MPDCompress on several network models, compression rates, and image datasets. On the LeNet 300-100 model (MNIST dataset), Deep MNIST, and CIFAR10, we achieve 10 X network compression with less than 1% accuracy loss compared to non-compressed accuracy performance. On AlexNet for the full ImageNet ILSVRC-2012 dataset, we achieve 8 X network compression with less than 1% accuracy loss, with top-5 and top-1 accuracies of 79.6% and 56.4%, respectively. Finally, we observe that the algorithm can offer inference speedups across various hardware platforms, with 4 X faster operation achieved on several mobile GPUs.
References in corpus (16)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Deep Residual Learning for Image Recognition
- DeepPose: Human Pose Estimation via Deep Neural Networks
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- Trained Ternary Quantization
- Compressing Neural Networks with the Hashing Trick
- Learning Structured Sparsity in Deep Neural Networks
- DSD: Dense-Sparse-Dense Training for Deep Neural Networks
- ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
- Structured Pruning of Deep Convolutional Neural Networks
- DropNeuron: Simplifying the Structure of Deep Neural Networks
- NoiseOut: A Simple Way to Prune Neural Networks
- SEP-Nets: Small and Effective Pattern Networks
- Structured Deep Neural Network Pruning via Matrix Pivoting