Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

Abstract

Dynamics of human body skeletons convey significant information for human action recognition. Conventional approaches for modeling skeletons usually rely on hand-crafted parts or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we propose a novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data. This formulation not only leads to greater expressive power but also stronger generalization capability. On two large datasets, Kinetics and NTU-RGBD, it achieves substantial improvements over mainstream methods.

Accepted by AAAI 2018

References in corpus (5)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• The Kinetics Human Action Video Dataset
• Deep Convolutional Networks on Graph-Structured Data
• Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
• Differential Recurrent Neural Networks for Action Recognition

Cited by in corpus (40)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• TSA-Net: Tube Self-Attention Network for Action Quality Assessment
• Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things
• Bipartite Graph Reasoning GANs for Person Image Generation
• 3D Hand Shape and Pose Estimation from a Single RGB Image
• SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction
• Action Machine: Rethinking Action Recognition in Trimmed Videos
• Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective
• GraFormer: Graph Convolution Transformer for 3D Pose Estimation
• Multi-Instrumentalist Net: Unsupervised Generation of Music from Body Movements
• LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking
• Spatio-temporal Action Recognition: A Survey
• Uncertainty-aware Score Distribution Learning for Action Quality Assessment
• Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition
• StressNet: Deep Learning to Predict Stress With Fracture Propagation in Brittle Materials
• JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition
• Iterate & Cluster: Iterative Semi-Supervised Action Recognition
• Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective
• Spectral Temporal Graph Neural Network for Trajectory Prediction
• A Graph Attention Based Approach for Trajectory Prediction in Multi-agent Sports Games
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• Temporal Relational Modeling with Self-Supervision for Action Segmentation
• SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition
• Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos
• Learning Hierarchical Graph Neural Networks for Image Clustering
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
• Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling
• LSTA-Net: Long short-term Spatio-Temporal Aggregation Network for Skeleton-based Action Recognition
• Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
• Learning Actor Relation Graphs for Group Activity Recognition
• Learning to Cluster Faces on an Affinity Graph
• Follow the Attention: Combining Partial Pose and Object Motion for Fine-Grained Action Detection
• Multi-grained Trajectory Graph Convolutional Networks for Habit-unrelated Human Motion Prediction
• UAV-GESTURE: A Dataset for UAV Control and Gesture Recognition
• Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
• Poisson Kernel Avoiding Self-Smoothing in Graph Convolutional Networks
• Angular Embedding: A New Angular Robust Principal Component Analysis
• Effective Human Activity Recognition Based on Small Datasets
• Better Understanding Hierarchical Visual Relationship for Image Caption
• Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions