Actor-Centric Relation Network

Abstract

Current state-of-the-art approaches for spatio-temporal action localization rely on detections at the frame level and model temporal context with 3D ConvNets. Here, we go one step further and model spatio-temporal relations to capture the interactions between human actors, relevant objects and scene elements essential to differentiate similar human actions. Our approach is weakly supervised and mines the relevant elements automatically with an actor-centric relational network (ACRN). ACRN computes and accumulates pair-wise relation information from actor and global scene features, and generates relation features for action classification. It is implemented as neural networks and can be trained jointly with an existing action detection system. We show that ACRN outperforms alternative approaches which capture relation information, and that the proposed framework improves upon the state-of-the-art performance on JHMDB and AVA. A visualization of the learned relation features confirms that our approach is able to attend to the relevant relations for each action.

arXiv commentECCV 2018 camera ready

References in corpus (8)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• The Kinetics Human Action Video Dataset
• ParseNet: Looking Wider to See Better
• Action Recognition using Visual Attention
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Visual Semantic Role Labeling
• Exploring Person Context and Local Scene Context for Object Detection

Cited by in corpus (7)

• Learning Discriminative Motion Features Through Detection
• ACDnet: An action detection network for real-time edge computing based on flow-guided feature approximation and memory aggregation
• Relational Action Forecasting
• SAFCAR: Structured Attention Fusion for Compositional Action Recognition
• Symbiotic Attention with Privileged Information for Egocentric Action Recognition
• PGGANet: Pose Guided Graph Attention Network for Person Re-identification
• Long Short-Term Relation Networks for Video Action Detection