From Hashing to CNNs: Training BinaryWeight Networks via Hashing

Abstract

Deep convolutional neural networks (CNNs) have shown appealing performance on various computer vision tasks in recent years. This motivates people to deploy CNNs to realworld applications. However, most of state-of-art CNNs require large memory and computational resources, which hinders the deployment on mobile devices. Recent studies show that low-bit weight representation can reduce much storage and memory demand, and also can achieve efficient network inference. To achieve this goal, we propose a novel approach named BWNH to train Binary Weight Networks via Hashing. In this paper, we first reveal the strong connection between inner-product preserving hashing and binary weight networks, and show that training binary weight networks can be intrinsically regarded as a hashing problem. Based on this perspective, we propose an alternating optimization method to learn the hash codes instead of directly learning binary weights. Extensive experiments on CIFAR10, CIFAR100 and ImageNet demonstrate that our proposed BWNH outperforms current state-of-art by a large margin.

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Compressing Deep Convolutional Networks using Vector Quantization
• Tensorizing Neural Networks
• Compressing Neural Networks with the Hashing Trick
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

Cited by in corpus (14)

• Binary Neural Networks: A Survey
• Pruning and Quantization for Deep Neural Network Acceleration: A Survey
• Accurate and Compact Convolutional Neural Networks with Trained Binarization
• Forward and Backward Information Retention for Accurate Binary Neural Networks
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Collaborative Learning for Extremely Low Bit Asymmetric Hashing
• Distribution-sensitive Information Retention for Accurate Binary Neural Network
• Disentangling Neural Architectures and Weights: A Case Study in Supervised Classification
• Iterative Training: Finding Binary Weight Deep Neural Networks with Layer Binarization
• A Comparative Study of Neural Network Compression
• Differentiable Architecture Pruning for Transfer Learning
• Exact Backpropagation in Binary Weighted Networks with Group Weight Transformations
• Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration
• A Targeted Acceleration and Compression Framework for Low bit Neural Networks