Towards High Performance Video Object Detection for Mobiles

Abstract

Despite the recent success of video object detection on Desktop GPUs, its architecture is still far too heavy for mobiles. It is also unclear whether the key principles of sparse feature propagation and multi-frame feature aggregation apply at very limited computational resources. In this paper, we present a light weight network architecture for video object detection on mobiles. Light weight image object detector is applied on sparse key frames. A very small network, Light Flow, is designed for establishing correspondence across frames. A flow-guided GRU module is designed to effectively aggregate features on key frames. For non-key frames, sparse feature propagation is performed. The whole network can be trained end-to-end. The proposed system achieves 60.2% mAP score at speed of 25.6 fps on mobiles (e.g., HuaWei Mate 8).

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Deep Residual Learning for Image Recognition
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Densely Connected Convolutional Networks
• Focal Loss for Dense Object Detection
• T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
• Feature Pyramid Networks for Object Detection
• YOLO9000: Better, Faster, Stronger
• Deformable Convolutional Networks
• Aggregated Residual Transformations for Deep Neural Networks
• Xception: Deep Learning with Depthwise Separable Convolutions
• Light-Head R-CNN: In Defense of Two-Stage Object Detector
• Delving Deeper into Convolutional Networks for Learning Video Representations
• Flow-Guided Feature Aggregation for Video Object Detection
• Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video
• VideoLSTM Convolves, Attends and Flows for Action Recognition
• Mobile Video Object Detection with Temporally-Aware Feature Maps
• Towards High Performance Video Object Detection

Cited by in corpus (12)

• Deep Learning for UAV-based Object Detection and Tracking: A Survey
• Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks
• MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection
• Looking Fast and Slow: Memory-Guided Mobile Video Object Detection
• Joint Detection and Tracking in Videos with Identification Features
• Recurrent Neural Networks for video object detection
• Fast Object Detection in Compressed Video
• Learning to Track Object Position through Occlusion
• Patchwork: A Patch-wise Attention Network for Efficient Object Detection and Segmentation in Video Streams
• Deep Learning Acceleration Techniques for Real Time Mobile Vision Applications
• Dynamic Fusion Network For Light Field Depth Estimation
• HMM-guided frame querying for bandwidth-constrained video search