Learning to track for spatio-temporal action localization
arXiv:1506.01929
Abstract
We propose an effective approach for spatio-temporal action localization in realistic videos. The approach first detects proposals at the frame-level and scores them with a combination of static and motion CNN features. It then tracks high-scoring proposals throughout the video using a tracking-by-detection approach. Our tracker relies simultaneously on instance-level and class-level detectors. The tracks are scored using a spatio-temporal motion histogram, a descriptor at the track level, in combination with the CNN features. Finally, we perform temporal localization of the action using a sliding-window approach at the track level. We present experimental results for spatio-temporal localization on the UCF-Sports, J-HMDB and UCF-101 action localization datasets, where our approach outperforms the state of the art with a margin of 15%, 7% and 12% respectively in mAP.
References in corpus (4)
Cited by in corpus (38)
- Convolutional Two-Stream Network Fusion for Video Action Recognition
- Temporal Action Detection with Structured Segment Networks
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Human Action Localization with Sparse Spatial Supervision
- VideoLSTM Convolves, Attends and Flows for Action Recognition
- CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
- Action Recognition with Image Based CNN Features
- Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos
- NoScope: Optimizing Neural Network Queries over Video at Scale
- End-to-end Learning of Action Detection from Frame Glimpses in Videos
- Two Stream LSTM: A Deep Fusion Framework for Human Action Recognition
- Spatio-Temporal Action Detection with Cascade Proposal and Location Anticipation
- Learning to Anonymize Faces for Privacy Preserving Action Detection
- Incremental Tube Construction for Human Action Detection
- Spatio-temporal Action Recognition: A Survey
- Localizing Actions from Video Labels and Pseudo-Annotations
- Real-Time End-to-End Action Detection with Two-Stream Networks
- AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecture
- Spatio-temporal Human Action Localisation and Instance Segmentation in Temporally Untrimmed Videos
- Relational Action Forecasting
- ReHAR: Robust and Efficient Human Activity Recognition
- DAP3D-Net: Where, What and How Actions Occur in Videos?
- Beyond Caption To Narrative: Video Captioning With Multiple Sentences
- Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking
- Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
- ActionVLAD: Learning spatio-temporal aggregation for action classification
- Spatio-Temporal Instance Learning: Action Tubes from Class Supervision
- Action Detection from a Robot-Car Perspective
- Action Tubelet Detector for Spatio-Temporal Action Localization
- Generic Tubelet Proposals for Action Localization
- Multilevel Language and Vision Integration for Text-to-Clip Retrieval
- Efficient Action Detection in Untrimmed Videos via Multi-Task Learning
- Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection
- Spatio-Temporal Action Localization in a Weakly Supervised Setting
- Discovering Spatio-Temporal Action Tubes
- Temporal Action Detection by Joint Identification-Verification
- Long Short-Term Relation Networks for Video Action Detection
- Spot On: Action Localization from Pointly-Supervised Proposals