Contextual Multi-Scale Region Convolutional 3D Network for Activity Detection
arXiv:1801.09184
Abstract
Activity detection is a fundamental problem in computer vision. Detecting activities of different temporal scales is particularly challenging. In this paper, we propose the contextual multi-scale region convolutional 3D network (CMS-RC3D) for activity detection. To deal with the inherent temporal scale variability of activity instances, the temporal feature pyramid is used to represent activities of different temporal scales. On each level of the temporal feature pyramid, an activity proposal detector and an activity classifier are learned to detect activities of specific temporal scales. Temporal contextual information is fused into activity classifiers for better recognition. More importantly, the entire model at all levels can be trained end-to-end. Our CMS-RC3D detector can deal with activities at all temporal scale ranges with only a single pass through the backbone network. We test our detector on two public activity detection benchmarks, THUMOS14 and ActivityNet. Extensive experiments show that the proposed CMS-RC3D detector outperforms state-of-the-art methods on THUMOS14 by a substantial margin and achieves comparable results on ActivityNet despite using a shallow feature extractor.
10 pages, 3 figures
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Feature Pyramid Networks for Object Detection
- Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
- Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Temporal Context Network for Activity Localization in Videos
Cited by in corpus (4)
- DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition
- HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization
- 3D Feature Pyramid Attention Module for Robust Visual Speech Recognition
- Class Semantics-based Attention for Action Detection