Fixed-point Factorized Networks

Abstract

In recent years, Deep Neural Networks (DNN) based methods have achieved remarkable performance in a wide range of tasks and have been among the most powerful and widely used techniques in computer vision. However, DNN-based methods are both computational-intensive and resource-consuming, which hinders the application of these methods on embedded systems like smart phones. To alleviate this problem, we introduce a novel Fixed-point Factorized Networks (FFN) for pretrained models to reduce the computational complexity as well as the storage requirement of networks. The resulting networks have only weights of -1, 0 and 1, which significantly eliminates the most resource-consuming multiply-accumulate operations (MACs). Extensive experiments on large-scale ImageNet classification task show the proposed FFN only requires one-thousandth of multiply operations with comparable accuracy.

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• FitNets: Hints for Thin Deep Nets
• Deep Learning with Limited Numerical Precision
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Convolutional Neural Networks using Logarithmic Data Representation
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Bitwise Neural Networks
• ImageNet pre-trained models with batch normalization
• Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation