Mobile Video Object Detection with Temporally-Aware Feature Maps
arXiv:1711.06368
Abstract
This paper introduces an online model for object detection in videos designed to run in real-time on low-powered mobile and embedded devices. Our approach combines fast single-image object detection with convolutional long short term memory (LSTM) layers to create an interweaved recurrent-convolutional architecture. Additionally, we propose an efficient Bottleneck-LSTM layer that significantly reduces computational cost compared to regular LSTMs. Our network achieves temporal awareness by using Bottleneck-LSTMs to refine and propagate feature maps across frames. This approach is substantially faster than existing detection methods in video, outperforming the fastest single-frame models in model size and computational cost while attaining accuracy comparable to much more expensive single-frame models on the Imagenet VID 2015 dataset. Our model reaches a real-time inference speed of up to 15 FPS on a mobile CPU.
arXiv commentIn CVPR 2018
References in corpus (23)
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Deep Residual Learning for Image Recognition
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- A Critical Review of Recurrent Neural Networks for Sequence Learning
- Going Deeper with Convolutions
- Focal Loss for Dense Object Detection
- T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Rich feature hierarchies for accurate object detection and semantic segmentation
- Xception: Deep Learning with Depthwise Separable Convolutions
- Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
- Spatio-temporal video autoencoder with differentiable memory
- Seq-NMS for Video Object Detection
- Flow-Guided Feature Aggregation for Video Object Detection
- PVANet: Lightweight Deep Neural Networks for Real-time Object Detection
- NoScope: Optimizing Neural Network Queries over Video at Scale
- Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking
- Quantized Convolutional Neural Networks for Mobile Devices
Cited by in corpus (7)
- Small-scale Pedestrian Detection Based on Somatic Topology Localization and Temporal Feature Aggregation
- Towards High Performance Video Object Detection for Mobiles
- Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation
- RetinaTrack: Online Single Stage Joint Detection and Tracking
- FASTER Recurrent Networks for Efficient Video Classification
- Fast Object Detection in Compressed Video
- PatchNet -- Short-range Template Matching for Efficient Video Processing