Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving

Abstract

Autonomous driving has harsh requirements of small model size and energy efficiency, in order to enable the embedded system to achieve real-time on-board object detection. Recent deep convolutional neural network based object detectors have achieved state-of-the-art accuracy. However, such models are trained with numerous parameters and their high computational costs and large storage prohibit the deployment to memory and computation resource limited systems. Low-precision neural networks are popular techniques for reducing the computation requirements and memory footprint. Among them, binary weight neural network (BWN) is the extreme case which quantizes the float-point into just $1$ bit. BWNs are difficult to train and suffer from accuracy deprecation due to the extreme low-bit representation. To address this problem, we propose a knowledge transfer (KT) method to aid the training of BWN using a full-precision teacher network. We built DarkNet- and MobileNet-based binary weight YOLO-v2 detectors and conduct experiments on KITTI benchmark for car, pedestrian and cyclist detection. The experimental results show that the proposed method maintains high detection accuracy while reducing the model size of DarkNet-YOLO from 257 MB to 8.8 MB and MobileNet-YOLO from 193 MB to 7.9 MB.

arXiv commentAccepted by ICRA 2019

References in corpus (12)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
• Training deep neural networks with low precision multiplications
• Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
• SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving
• Ternary Neural Networks with Fine-Grained Quantization
• ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks