Learning Actor Relation Graphs for Group Activity Recognition

Abstract

Modeling relation between actors is important for recognizing group activity in a multi-person scene. This paper aims at learning discriminative relation between actors efficiently using deep models. To this end, we propose to build a flexible and efficient Actor Relation Graph (ARG) to simultaneously capture the appearance and position relation between actors. Thanks to the Graph Convolutional Network, the connections in ARG could be automatically learned from group activity videos in an end-to-end manner, and the inference on ARG could be efficiently performed with standard matrix operations. Furthermore, in practice, we come up with two variants to sparsify ARG for more effective modeling in videos: spatially localized ARG and temporal randomized ARG. We perform extensive experiments on two standard group activity recognition datasets: the Volleyball dataset and the Collective Activity dataset, where state-of-the-art performance is achieved on both datasets. We also visualize the learned actor graphs and relation features, which demonstrate that the proposed ARG is able to capture the discriminative relation information for group activity recognition.

Accepted by CVPR 2019

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Inductive Representation Learning on Large Graphs
• Learning Convolutional Neural Networks for Graphs
• Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
• A simple neural network module for relational reasoning
• Learning Multiagent Communication with Backpropagation
• Scene Graph Generation by Iterative Message Passing
• Relation Networks for Object Detection
• Detecting Visual Relationships with Deep Relational Networks
• Learning to Detect Human-Object Interactions
• Modeling Relationships in Referential Expressions with Compositional Modular Networks
• Detecting events and key actors in multi-person videos
• Structure Inference Machines: Recurrent Neural Networks for Analyzing Relations in Group Activity Recognition
• Actions and Attributes from Wholes and Parts