Vertex Feature Encoding and Hierarchical Temporal Modeling in a Spatial-Temporal Graph Convolutional Network for Action Recognition

Abstract

This paper extends the Spatial-Temporal Graph Convolutional Network (ST-GCN) for skeleton-based action recognition by introducing two novel modules, namely, the Graph Vertex Feature Encoder (GVFE) and the Dilated Hierarchical Temporal Convolutional Network (DH-TCN). On the one hand, the GVFE module learns appropriate vertex features for action recognition by encoding raw skeleton data into a new feature space. On the other hand, the DH-TCN module is capable of capturing both short-term and long-term temporal dependencies using a hierarchical dilated convolutional network. Experiments have been conducted on the challenging NTU RGB-D-60 and NTU RGB-D 120 datasets. The obtained results show that our method competes with state-of-the-art approaches while using a smaller number of layers and parameters; thus reducing the required training time and memory.

References in corpus (3)

• Spectral Networks and Locally Connected Networks on Graphs
• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition

Cited by in corpus (5)

• Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition
• Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognition
• Feedback Graph Convolutional Network for Skeleton-based Action Recognition
• Face-GCN: A Graph Convolutional Network for 3D Dynamic Face Identification/Recognition
• Hierarchical Action Classification with Network Pruning