A Multi-Stream Convolutional Neural Network Framework for Group Activity Recognition
arXiv:1812.10328
Abstract
In this work, we present a framework based on multi-stream convolutional neural networks (CNNs) for group activity recognition. Streams of CNNs are separately trained on different modalities and their predictions are fused at the end. Each stream has two branches to predict the group activity based on person and scene level representations. A new modality based on the human pose estimation is presented to add extra information to the model. We evaluate our method on the Volleyball and Collective Activity datasets. Experimental results show that the proposed framework is able to achieve state-of-the-art results when multiple or single frames are given as input to the model with 90.50% and 86.61% accuracy on Volleyball dataset, respectively, and 87.01% accuracy of multiple frames group activity on Collective Activity dataset.
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Rethinking the Inception Architecture for Computer Vision
- Convolutional Two-Stream Network Fusion for Video Action Recognition
- Convolutional Pose Machines
- Detecting events and key actors in multi-person videos
- Structure Inference Machines: Recurrent Neural Networks for Analyzing Relations in Group Activity Recognition
- Visual Recognition by Counting Instances: A Multi-Instance Cardinality Potential Kernel
- Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
Cited by in corpus (7)
- Skeleton-based Relational Reasoning for Group Activity Analysis
- Pose is all you need: The pose only group activity recognition system (POGARS)
- Learning Group Activities from Skeletons without Individual Action Labels
- L-CNN: A Lattice cross-fusion strategy for multistream convolutional neural networks
- Streaming Networks: Increase Noise Robustness and Filter Diversity via Hard-wired and Input-induced Sparsity
- Applications of the Streaming Networks
- Sparsifying and Down-scaling Networks to Increase Robustness to Distortions