Decoupling Localization and Classification in Single Shot Temporal Action Detection
arXiv:1904.07442
Abstract
Video temporal action detection aims to temporally localize and recognize the action in untrimmed videos. Existing one-stage approaches mostly focus on unifying two subtasks, i.e., localization of action proposals and classification of each proposal through a fully shared backbone. However, such design of encapsulating all components of two subtasks in one single network might restrict the training by ignoring the specialized characteristic of each subtask. In this paper, we propose a novel Decoupled Single Shot temporal Action Detection (Decouple-SSAD) method to mitigate such problem by decoupling the localization and classification in a one-stage scheme. Particularly, two separate branches are designed in parallel to enable each component to own representations privately for accurate localization or classification. Each branch produces a set of action anchor layers by applying deconvolution to the feature maps of the main stream. Each branch produces a set of feature maps by applying deconvolution to the feature maps of the main stream. High-level semantic information from deeper layers is thus incorporated to enhance the feature representations. We conduct extensive experiments on THUMOS14 dataset and demonstrate superior performance over state-of-the-art methods. Our code is available online.
ICME 2019. https://github.com/HYPJUDY/Decouple-SSAD
References in corpus (9)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
- Temporal Action Detection with Structured Segment Networks
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Temporal Context Network for Activity Localization in Videos
- TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
- UntrimmedNets for Weakly Supervised Action Recognition and Detection