Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning

Abstract

Skeleton-based action recognition has made great progress recently, but many problems still remain unsolved. For example, most of the previous methods model the representations of skeleton sequences without abundant spatial structure information and detailed temporal dynamics features. In this paper, we propose a novel model with spatial reasoning and temporal stack learning (SR-TSL) for skeleton based action recognition, which consists of a spatial reasoning network (SRN) and a temporal stack learning network (TSLN). The SRN can capture the high-level spatial structural information within each frame by a residual graph neural network, while the TSLN can model the detailed temporal dynamics of skeleton sequences by a composition of multiple skip-clip LSTMs. During training, we propose a clip-based incremental loss to optimize the model. We perform extensive experiments on the SYSU 3D Human-Object Interaction dataset and NTU RGB+D dataset and verify the effectiveness of each network of our model. The comparison results illustrate that our approach achieves much better results than state-of-the-art methods.

ECCV 2018

References in corpus (6)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Deep Convolutional Networks on Graph-Structured Data
• Learning Convolutional Neural Networks for Graphs
• An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Situation Recognition with Graph Neural Networks

Cited by in corpus (8)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition
• Analyzing Human-Human Interactions: A Survey
• Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching
• StressNet: Deep Learning to Predict Stress With Fracture Propagation in Brittle Materials
• JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition