Learning Group Activities from Skeletons without Individual Action Labels

Abstract

To understand human behavior we must not just recognize individual actions but model possibly complex group activity and interactions. Hierarchical models obtain the best results in group activity recognition but require fine grained individual action annotations at the actor level. In this paper we show that using only skeletal data we can train a state-of-the art end-to-end system using only group activity labels at the sequence level. Our experiments show that models trained without individual action supervision perform poorly. On the other hand we show that pseudo-labels can be computed from any pre-trained feature extractor with comparable final performance. Finally our carefully designed lean pose only architecture shows highly competitive results versus more complex multimodal approaches even in the self-supervised variant.

arXiv commentICPR 2020

References in corpus (7)

• The Kinetics Human Action Video Dataset
• An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
• Deep Clustering for Unsupervised Learning of Visual Features
• Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation
• A Multi-Stream Convolutional Neural Network Framework for Group Activity Recognition
• Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
• Convolutional Relational Machine for Group Activity Recognition