Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

Abstract

Dynamics of human body skeletons convey significant information for human action recognition. Conventional approaches for modeling skeletons usually rely on hand-crafted parts or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we propose a novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data. This formulation not only leads to greater expressive power but also stronger generalization capability. On two large datasets, Kinetics and NTU-RGBD, it achieves substantial improvements over mainstream methods.

Accepted by AAAI 2018

References in corpus (12)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• The Kinetics Human Action Video Dataset
• Spectral Networks and Locally Connected Networks on Graphs
• Pixel Recurrent Neural Networks
• Deep Convolutional Networks on Graph-Structured Data
• Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
• Learning Convolutional Neural Networks for Graphs
• Deformable Convolutional Networks
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
• Temporal Action Detection with Structured Segment Networks
• Differential Recurrent Neural Networks for Action Recognition

Cited by in corpus (73)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Spatial Temporal Transformer Network for Skeleton-based Action Recognition
• Graph Convolutional Networks for Graphs Containing Missing Features
• STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
• ARBEE: Towards Automated Recognition of Bodily Expression of Emotion In the Wild
• TSA-Net: Tube Self-Attention Network for Action Quality Assessment
• A Survey on 3D Skeleton-Based Action Recognition Using Learning Method
• Synthetic Humans for Action Recognition from Unseen Viewpoints
• Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things
• Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences
• Identifying Emotions from Walking using Affective and Deep Features
• Graph Convolutional Networks for Temporal Action Localization
• Bipartite Graph Reasoning GANs for Person Image Generation
• Human Action Performance using Deep Neuro-Fuzzy Recurrent Attention Model
• 3D Hand Shape and Pose Estimation from a Single RGB Image
• MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
• SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction
• Make Skeleton-based Action Recognition Model Smaller, Faster and Better
• Action Machine: Rethinking Action Recognition in Trimmed Videos
• Adversarial Attack Framework on Graph Embedding Models with Limited Knowledge
• A Review of Graph Neural Networks and Their Applications in Power Systems
• Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective
• Graph Edge Convolutional Neural Networks for Skeleton Based Action Recognition
• UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles
• Physically-interpretable classification of biological network dynamics for complex collective motions
• Person Re-identification with Deep Similarity-Guided Graph Neural Network
• GraFormer: Graph Convolution Transformer for 3D Pose Estimation
• Multi-Instrumentalist Net: Unsupervised Generation of Music from Body Movements
• Spectral Feature Transformation for Person Re-identification
• LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking
• Spatio-temporal Action Recognition: A Survey
• EV-Action: Electromyography-Vision Multi-Modal Action Dataset
• Graph Signal Processing for Geometric Data and Beyond: Theory and Applications
• Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition
• StressNet: Deep Learning to Predict Stress With Fracture Propagation in Brittle Materials
• Uncertainty-aware Score Distribution Learning for Action Quality Assessment
• JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition
• Iterate & Cluster: Iterative Semi-Supervised Action Recognition
• RGB-D Based Action Recognition with Light-weight 3D Convolutional Networks
• Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective
• A Graph Attention Based Approach for Trajectory Prediction in Multi-agent Sports Games
• Spectral Temporal Graph Neural Network for Trajectory Prediction
• Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
• Temporal Relational Modeling with Self-Supervision for Action Segmentation
• Quaternion Product Units for Deep Learning on 3D Rotation Groups
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
• Learning Hierarchical Graph Neural Networks for Image Clustering
• Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos
• Temporal Reasoning Graph for Activity Recognition
• Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling
• SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition
• Learning to Cluster Faces on an Affinity Graph
• WildGait: Learning Gait Representations from Raw Surveillance Streams
• A hybrid quantum-classical neural network with deep residual learning
• Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
• LSTA-Net: Long short-term Spatio-Temporal Aggregation Network for Skeleton-based Action Recognition
• Follow the Attention: Combining Partial Pose and Object Motion for Fine-Grained Action Detection
• Multi-grained Trajectory Graph Convolutional Networks for Habit-unrelated Human Motion Prediction
• UAV-GESTURE: A Dataset for UAV Control and Gesture Recognition
• Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
• Learning Actor Relation Graphs for Group Activity Recognition
• Learning Chebyshev Basis in Graph Convolutional Networks for Skeleton-based Action Recognition
• Poisson Kernel Avoiding Self-Smoothing in Graph Convolutional Networks
• Graph Feature Gating Networks
• Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions
• Effective Human Activity Recognition Based on Small Datasets
• Better Understanding Hierarchical Visual Relationship for Image Caption
• Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition
• Anomaly Detection for Aggregated Data Using Multi-Graph Autoencoder
• Angular Embedding: A New Angular Robust Principal Component Analysis
• NTU-X: An Enhanced Large-scale Dataset for Improving Pose-based Recognition of Subtle Human Actions