Spatio-Temporal Action Detection with Cascade Proposal and Location Anticipation

Abstract

In this work, we address the problem of spatio-temporal action detection in temporally untrimmed videos. It is an important and challenging task as finding accurate human actions in both temporal and spatial space is important for analyzing large-scale video data. To tackle this problem, we propose a cascade proposal and location anticipation (CPLA) model for frame-level action detection. There are several salient points of our model: (1) a cascade region proposal network (casRPN) is adopted for action proposal generation and shows better localization accuracy compared with single region proposal network (RPN); (2) action spatio-temporal consistencies are exploited via a location anticipation network (LAN) and thus frame-level action detection is not conducted independently. Frame-level detections are then linked by solving an linking score maximization problem, and temporally trimmed into spatio-temporal action tubes. We demonstrate the effectiveness of our model on the challenging UCF101 and LIRIS-HARL datasets, both achieving state-of-the-art performance.

arXiv commentAccepted at BMVC 2017 (oral)

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
• Learning to track for spatio-temporal action localization
• Contextual Action Recognition with R*CNN

Cited by in corpus (10)

• Incremental Tube Construction for Human Action Detection
• Mutual Suppression Network for Video Prediction using Disentangled Features
• Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding
• Spatio-Temporal Instance Learning: Action Tubes from Class Supervision
• Action Detection from a Robot-Car Perspective
• Dance with Flow: Two-in-One Stream Action Detection
• Follow the Attention: Combining Partial Pose and Object Motion for Fine-Grained Action Detection
• Online Detection of Action Start in Untrimmed, Streaming Videos
• Long Short-Term Relation Networks for Video Action Detection
• Finding Action Tubes with a Sparse-to-Dense Framework