Binarized Neural Networks on the ImageNet Classification Task

Abstract

We trained Binarized Neural Networks (BNNs) on the high resolution ImageNet ILSVRC-2102 dataset classification task and achieved a good performance. With a moderate size network of 13 layers, we obtained top-5 classification accuracy rate of 84.1 % on validation set through network distillation, much better than previous published results of 73.2% on XNOR network and 69.1% on binarized GoogleNET. We expect networks of better performance can be obtained by following our current strategies. We provide a detailed discussion and preliminary analysis on strategies used in the network training.

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Compressing Neural Networks with the Hashing Trick
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• Efficient supervised learning in networks with binary synapses

Cited by in corpus (3)

• What does it mean to understand a neural network?
• FQ-Conv: Fully Quantized Convolution for Efficient and Accurate Inference
• Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer