AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling

Abstract

In vision-enabled autonomous systems such as robots and autonomous cars, video object detection plays a crucial role, and both its speed and accuracy are important factors to provide reliable operation. The key insight we show in this paper is that speed and accuracy are not necessarily a trade-off when it comes to image scaling. Our results show that re-scaling the image to a lower resolution will sometimes produce better accuracy. Based on this observation, we propose a novel approach, dubbed AdaScale, which adaptively selects the input image scale that improves both accuracy and speed for video object detection. To this end, our results on ImageNet VID and mini YouTube-BoundingBoxes datasets demonstrate 1.3 points and 2.7 points mAP improvement with 1.6x and 1.8x speedup, respectively. Additionally, we improve state-of-the-art video acceleration work by an extra 1.25x speedup with slightly better mAP on ImageNet VID dataset.

Accepted to SysML 2019 (http://www.sysml.cc/) as oral contribution

References in corpus (11)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Focal Loss for Dense Object Detection
• T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
• Feature Pyramid Networks for Object Detection
• YOLO9000: Better, Faster, Stronger
• Deformable Convolutional Networks
• Seq-NMS for Video Object Detection
• YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
• Towards High Performance Video Object Detection

Cited by in corpus (6)

• TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers
• Regularizing Activation Distribution for Training Binarized Deep Networks
• MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution
• OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation
• E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings
• Enabling Incremental Knowledge Transfer for Object Detection at the Edge