Finding Action Tubes with a Sparse-to-Dense Framework
arXiv:2008.13196
Abstract
The task of spatial-temporal action detection has attracted increasing attention among researchers. Existing dominant methods solve this problem by relying on short-term information and dense serial-wise detection on each individual frames or clips. Despite their effectiveness, these methods showed inadequate use of long-term information and are prone to inefficiency. In this paper, we propose for the first time, an efficient framework that generates action tube proposals from video streams with a single forward pass in a sparse-to-dense manner. There are two key characteristics in this framework: (1) Both long-term and short-term sampled information are explicitly utilized in our spatiotemporal network, (2) A new dynamic feature sampling module (DTS) is designed to effectively approximate the tube output while keeping the system tractable. We evaluate the efficacy of our model on the UCF101-24, JHMDB-21 and UCFSports benchmark datasets, achieving promising results that are competitive to state-of-the-art methods. The proposed sparse-to-dense strategy rendered our framework about 7.6 times more efficient than the nearest competitor.
5 figures; AAAI 2020
References in corpus (4)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- An End-to-end 3D Convolutional Neural Network for Action Detection and Segmentation in Videos
- Spatio-Temporal Action Detection with Cascade Proposal and Location Anticipation