Evolving Skeletons: Motion Dynamics in Action Recognition

Abstract

Skeleton-based action recognition has gained significant attention for its ability to efficiently represent spatiotemporal information in a lightweight format. Most existing approaches use graph-based models to process skeleton sequences, where each pose is represented as a skeletal graph structured around human physical connectivity. Among these, the Spatiotemporal Graph Convolutional Network (ST-GCN) has become a widely used framework. Alternatively, hypergraph-based models, such as the Hyperformer, capture higher-order correlations, offering a more expressive representation of complex joint interactions. A recent advancement, termed Taylor Videos, introduces motion-enhanced skeleton sequences by embedding motion concepts, providing a fresh perspective on interpreting human actions in skeleton-based action recognition. In this paper, we conduct a comprehensive evaluation of both traditional skeleton sequences and Taylor-transformed skeletons using ST-GCN and Hyperformer models on the NTU-60 and NTU-120 datasets. We compare skeletal graph and hypergraph representations, analyzing static poses against motion-injected poses. Our findings highlight the strengths and limitations of Taylor-transformed skeletons, demonstrating their potential to enhance motion dynamics while exposing current challenges in fully using their benefits. This study underscores the need for innovative skeletal modelling techniques to effectively handle motion-rich data and advance the field of action recognition.

Accepted at the Companion Proceedings of the ACM Web Conference (WWW Companion 2025)

References in corpus (10)

• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• A Comparative Review of Recent Kinect-based Action Recognition Algorithms
• Skeleton-Based Action Segmentation with Multi-Stage Spatial-Temporal Graph Convolutional Neural Networks
• Tensor Representations for Action Recognition
• A Survey on Hypergraph Neural Networks: An In-Depth and Step-By-Step Guide
• Self-supervising Action Recognition by Statistical Moment and Subspace Descriptors
• Hypergraph-based Multi-View Action Recognition using Event Cameras
• SATO: Stable Text-to-Motion Framework
• Deep Heterogeneous Contrastive Hyper-Graph Learning for In-the-Wild Context-Aware Human Activity Recognition
• SpATr: MoCap 3D Human Action Recognition based on Spiral Auto-encoder and Transformer Network