Fast Object Detection in Compressed Video

Abstract

Object detection in videos has drawn increasing attention since it is more practical in real scenarios. Most of the deep learning methods use CNNs to process each decoded frame in a video stream individually. However, the free of charge yet valuable motion information already embedded in the video compression format is usually overlooked. In this paper, we propose a fast object detection method by taking advantage of this with a novel Motion aided Memory Network (MMNet). The MMNet has two major advantages: 1) It significantly accelerates the procedure of feature extraction for compressed videos. It only need to run a complete recognition network for I-frames, i.e. a few reference frames in a video, and it produces the features for the following P frames (predictive frames) with a light weight memory network, which runs fast; 2) Unlike existing methods that establish an additional network to model motion of frames, we take full advantage of both motion vectors and residual errors that are freely available in video streams. To our best knowledge, the MMNet is the first work that investigates a deep convolutional detector on compressed videos. Our method is evaluated on the large-scale ImageNet VID dataset, and the results show that it is 3x times faster than single image detector R-FCN and 10x times faster than high-performance detector MANet at a minor accuracy loss.

10 pages, 7 figures and 2 tables

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Instance Normalization: The Missing Ingredient for Fast Stylization
• T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
• Seq-NMS for Video Object Detection
• Flow-Guided Feature Aggregation for Video Object Detection
• Mobile Video Object Detection with Temporally-Aware Feature Maps
• Towards High Performance Video Object Detection for Mobiles
• Object Detection in Video with Spatiotemporal Sampling Networks
• Dual Path Networks

Cited by in corpus (4)

• Fast object detection in compressed JPEG Images
• Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation
• Self-supervised Video Representation Learning by Context and Motion Decoupling
• PatchNet -- Short-range Template Matching for Efficient Video Processing