An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Abstract

Skeleton-based action recognition is an important task that requires the adequate understanding of movement characteristics of a human action from the given skeleton sequence. Recent studies have shown that exploring spatial and temporal features of the skeleton sequence is vital for this task. Nevertheless, how to effectively extract discriminative spatial and temporal features is still a challenging problem. In this paper, we propose a novel Attention Enhanced Graph Convolutional LSTM Network (AGC-LSTM) for human action recognition from skeleton data. The proposed AGC-LSTM can not only capture discriminative features in spatial configuration and temporal dynamics but also explore the co-occurrence relationship between spatial and temporal domains. We also present a temporal hierarchical architecture to increases temporal receptive fields of the top AGC-LSTM layer, which boosts the ability to learn the high-level semantic representation and significantly reduces the computation cost. Furthermore, to select discriminative spatial information, the attention mechanism is employed to enhance information of key joints in each AGC-LSTM layer. Experimental results on two datasets are provided: NTU RGB+D dataset and Northwestern-UCLA dataset. The comparison results demonstrate the effectiveness of our approach and show that our approach outperforms the state-of-the-art methods on both datasets.

Accepted by CVPR2019

References in corpus (9)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Deep Convolutional Networks on Graph-Structured Data
• An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Part-based Graph Convolutional Network for Action Recognition
• Learning Human-Object Interactions by Graph Parsing Neural Networks
• View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data
• Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning
• Situation Recognition with Graph Neural Networks

Cited by in corpus (13)

• Analyzing Human-Human Interactions: A Survey
• PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
• Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• Interpreting and Unifying Graph Neural Networks with An Optimization Framework
• Exploiting the ConvLSTM: Human Action Recognition using Raw Depth Video-Based Recurrent Neural Networks
• MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction
• PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation
• Hierarchical growing grid networks for skeleton based action recognition
• JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition
• Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• Quaternion Product Units for Deep Learning on 3D Rotation Groups
• LSTA-Net: Long short-term Spatio-Temporal Aggregation Network for Skeleton-based Action Recognition