ACDnet: An action detection network for real-time edge computing based on flow-guided feature approximation and memory aggregation

Abstract

Interpreting human actions requires understanding the spatial and temporal context of the scenes. State-of-the-art action detectors based on Convolutional Neural Network (CNN) have demonstrated remarkable results by adopting two-stream or 3D CNN architectures. However, these methods typically operate in a non-real-time, ofline fashion due to system complexity to reason spatio-temporal information. Consequently, their high computational cost is not compliant with emerging real-world scenarios such as service robots or public surveillance where detection needs to take place at resource-limited edge devices. In this paper, we propose ACDnet, a compact action detection network targeting real-time edge computing which addresses both efficiency and accuracy. It intelligently exploits the temporal coherence between successive video frames to approximate their CNN features rather than naively extracting them. It also integrates memory feature aggregation from past video frames to enhance current detection stability, implicitly modeling long temporal cues over time. Experiments conducted on the public benchmark datasets UCF-24 and JHMDB-21 demonstrate that ACDnet, when integrated with the SSD detector, can robustly achieve detection well above real-time (75 FPS). At the same time, it retains reasonable accuracy (70.92 and 49.53 frame mAP) compared to other top-performing methods using far heavier configurations. Codes will be available at https://github.com/dginhac/ACDnet.

Accepted for publication in Pattern Recognition Letters

References in corpus (12)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• FlowNet: Learning Optical Flow with Convolutional Networks
• T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
• Seq-NMS for Video Object Detection
• You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localization
• Impression Network for Video Object Detection
• Fast Optical Flow using Dense Inverse Search
• Actor-Centric Relation Network
• Actions as Moving Points

Cited by in corpus (1)

• A Semantic and Motion-Aware Spatiotemporal Transformer Network for Action Detection