Binarized Weight Error Networks With a Transition Regularization Term

Abstract

This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account the approximation error with an additional term. This model increases representation capacity and stability, particularly for shallow networks, while the computation load is theoretically reduced. In addition, a novel regularization term is introduced that is suitable for all threshold-based binary precision networks. This term penalizes the trainable parameters that are far from the thresholds at which binary transitions occur. This step promotes a swift modification for binary-precision responses at train time. The experimental results are carried out for two sets of tasks: visual classification and visual inverse problems. Benchmarks for Cifar10, SVHN, Fashion, ImageNet2012, Set5, Set14, Urban and BSD100 datasets show that our method outperforms all counterparts with binary precision.

arXiv commentSubmitted to ICIP 2021

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Ternary Weight Networks
• Trained Ternary Quantization
• Binary Neural Networks: A Survey
• LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
• Resource-Efficient Neural Networks for Embedded Systems