Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
arXiv:2111.03993
Abstract
Skeleton data is of low dimension. However, there is a trend of using very deep and complicated feedforward neural networks to model the skeleton sequence without considering the complexity in recent year. In this paper, a simple yet effective multi-scale semantics-guided neural network (MS-SGN) is proposed for skeleton-based action recognition. We explicitly introduce the high level semantics of joints (joint type and frame index) into the network to enhance the feature representation capability of joints. Moreover, a multi-scale strategy is proposed to be robust to the temporal scale variations. In addition, we exploit the relationship of joints hierarchically through two modules, i.e., a joint-level module for modeling the correlations of joints in the same frame and a frame-level module for modeling the temporal dependencies of frames. With an order of magnitude smaller model size than most previous methods, MSSGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
References in corpus (13)
- Semi-Supervised Classification with Graph Convolutional Networks
- NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
- Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
- An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
- The Microsoft 2017 Conversational Speech Recognition System
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
- An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition
- Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation
- View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data
- Learning Deep Bilinear Transformation for Fine-grained Image Representation
- Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning
- Space-Time Representation of People Based on 3D Skeletal Data: A Review
- Adding Attentiveness to the Neurons in Recurrent Neural Networks