SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems

Abstract

Object detection and tracking are challenging tasks for resource-constrained embedded systems. While these tasks are among the most compute-intensive tasks from the artificial intelligence domain, they are only allowed to use limited computation and memory resources on embedded devices. In the meanwhile, such resource-constrained implementations are often required to satisfy additional demanding requirements such as real-time response, high-throughput performance, and reliable inference accuracy. To overcome these challenges, we propose SkyNet, a hardware-efficient neural network to deliver the state-of-the-art detection accuracy and speed for embedded systems. Instead of following the common top-down flow for compact DNN (Deep Neural Network) design, SkyNet provides a bottom-up DNN design approach with comprehensive understanding of the hardware constraints at the very beginning to deliver hardware-efficient DNNs. The effectiveness of SkyNet is demonstrated by winning the competitive System Design Contest for low power object detection in the 56th IEEE/ACM Design Automation Conference (DAC-SDC), where our SkyNet significantly outperforms all other 100+ competitors: it delivers 0.731 Intersection over Union (IoU) and 67.33 frames per second (FPS) on a TX2 embedded GPU; and 0.716 IoU and 25.05 FPS on an Ultra96 embedded FPGA. The evaluation of SkyNet is also extended to GOT-10K, a recent large-scale high-diversity benchmark for generic object tracking in the wild. For state-of-the-art object trackers SiamRPN++ and SiamMask, where ResNet-50 is employed as the backbone, implementations using our SkyNet as the backbone DNN are 1.60X and 1.73X faster with better or similar accuracy when running on a 1080Ti GPU, and 37.20X smaller in terms of parameter size for significantly better memory and storage footprint.

Published as a conference paper at Conference on Machine Learning and Systems (MLSys) 2020

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Neural Architecture Search with Reinforcement Learning
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
• Ternary Weight Networks
• Hello Edge: Keyword Spotting on Microcontrollers
• YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
• CornerNet: Detecting Objects as Paired Keypoints
• Cascade R-CNN: High Quality Object Detection and Instance Segmentation
• Decoupled Classification Refinement: Hard False Positive Suppression for Object Detection
• Improving Object Detection from Scratch via Gated Feature Reuse
• SkyNet: A Champion Model for DAC-SDC on Low Power Object Detection
• StrassenNets: Deep Learning with a Multiplication Budget
• Single-Path NAS: Device-Aware Efficient ConvNet Design
• Ternary Hybrid Neural-Tree Networks for Highly Constrained IoT Applications

Cited by in corpus (13)

• AutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICs
• E${}^3$-UAV: An Edge-based Energy-Efficient Object Detection System for Unmanned Aerial Vehicles
• Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis
• DNNExplorer: A Framework for Modeling and Exploring a Novel Paradigm of FPGA-based DNN Accelerator
• MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions
• Any-Precision Deep Neural Networks
• Pseudo-IoU: Improving Label Assignment in Anchor-Free Object Detection
• EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search
• WinoCNN: Kernel Sharing Winograd Systolic Array for Efficient Convolutional Neural Network Acceleration on FPGAs
• Generic Neural Architecture Search via Regression
• RSCA: Real-time Segmentation-based Context-Aware Scene Text Detection
• Effective Algorithm-Accelerator Co-design for AI Solutions on Edge Devices
• Scaling Up Hardware Accelerator Verification using A-QED with Functional Decomposition