Efficient Action Detection in Untrimmed Videos via Multi-Task Learning
arXiv:1612.07403
Abstract
This paper studies the joint learning of action recognition and temporal localization in long, untrimmed videos. We employ a multi-task learning framework that performs the three highly related steps of action proposal, action recognition, and action localization refinement in parallel instead of the standard sequential pipeline that performs the steps in order. We develop a novel temporal actionness regression module that estimates what proportion of a clip contains action. We use it for temporal localization but it could have other applications like video retrieval, surveillance, summarization, etc. We also introduce random shear augmentation during training to simulate viewpoint change. We evaluate our framework on three popular video benchmarks. Results demonstrate that our joint model is efficient in terms of storage and computation in that we do not need to compute and cache dense trajectory features, and that it is several times faster than its sequential ConvNets counterpart. Yet, despite being more efficient, it outperforms state-of-the-art methods with respect to accuracy.
WACV 2017 camera ready, minor updates about test time efficiency
References in corpus (10)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Towards Good Practices for Very Deep Two-Stream ConvNets
- Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
- Temporal Localization of Fine-Grained Actions in Videos by Domain Transfer from Web Images
- DeepStereo: Learning to Predict New Views from the World's Imagery
- Learning to track for spatio-temporal action localization
- Domain Generalization for Object Recognition with Multi-task Autoencoders
- Objects2action: Classifying and localizing actions without any video example
- Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition
- DAP3D-Net: Where, What and How Actions Occur in Videos?