Optimizing Video Object Detection via a Scale-Time Lattice

Abstract

High-performance object detection relies on expensive convolutional networks to compute features, often leading to significant challenges in applications, e.g. those that require detecting objects from video streams in real time. The key to this problem is to trade accuracy for efficiency in an effective way, i.e. reducing the computing cost while maintaining competitive performance. To seek a good balance, previous efforts usually focus on optimizing the model architectures. This paper explores an alternative approach, that is, to reallocate the computation over a scale-time space. The basic idea is to perform expensive detection sparsely and propagate the results across both scales and time with substantially cheaper networks, by exploiting the strong correlations among them. Specifically, we present a unified framework that integrates detection, temporal propagation, and across-scale refinement on a Scale-Time Lattice. On this framework, one can explore various strategies to balance performance and cost. Taking advantage of this flexibility, we further develop an adaptive scheme with the detector invoked on demand and thus obtain improved tradeoff. On ImageNet VID dataset, the proposed method can achieve a competitive mAP 79.6% at 20 fps, or 79.0% at 62 fps as a performance/speed tradeoff.

arXiv commentAccepted to CVPR 2018. Project page: http://mmlab.ie.cuhk.edu.hk/projects/ST-Lattice/

References in corpus (12)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Feature Pyramid Networks for Object Detection
• YOLO9000: Better, Faster, Stronger
• Seq-NMS for Video Object Detection
• Flow-Guided Feature Aggregation for Video Object Detection
• DSOD: Learning Deeply Supervised Object Detectors from Scratch
• Watch and Learn: Semi-Supervised Learning of Object Detectors from Videos
• Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
• Discover and Learn New Objects from Documentaries

Cited by in corpus (4)

• QueryProp: Object Query Propagation for High-Performance Video Object Detection
• Transferable Adversarial Attacks for Image and Video Object Detection
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• CaTDet: Cascaded Tracked Detector for Efficient Object Detection from Video