SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition

Abstract

Due to the availability of large-scale skeleton datasets, 3D human action recognition has recently called the attention of computer vision community. Many works have focused on encoding skeleton data as skeleton image representations based on spatial structure of the skeleton joints, in which the temporal dynamics of the sequence is encoded as variations in columns and the spatial structure of each frame is represented as rows of a matrix. To further improve such representations, we introduce a novel skeleton image representation to be used as input of Convolutional Neural Networks (CNNs), named SkeleMotion. The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Different temporal scales are employed to compute motion values to aggregate more temporal dynamics to the representation making it able to capture longrange joint interactions involved in actions as well as filtering noisy motion values. Experimental results demonstrate the effectiveness of the proposed representation on 3D action recognition outperforming the state-of-the-art on NTU RGB+D 120 dataset.

arXiv comment16-th IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS2019)

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• A New Representation of Skeleton Sequences for 3D Action Recognition
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation
• View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Cited by in corpus (20)

• Human Action Recognition from Various Data Modalities: A Review
• Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition
• Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognition
• A Survey on 3D Skeleton-Based Action Recognition Using Learning Method
• PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
• Deep Independently Recurrent Neural Network (IndRNN)
• Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition
• Vertex Feature Encoding and Hierarchical Temporal Modeling in a Spatial-Temporal Graph Convolutional Network for Action Recognition
• Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition
• Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition
• Measuring Student Behavioral Engagement using Histogram of Actions
• Health Guardian: Using Multi-modal Data to Understand Individual Health
• Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition
• Signal-SGN: A Spiking Graph Convolutional Network for Skeletal Action Recognition via Learning Temporal-Frequency Dynamics
• Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment
• SkelVIT: Consensus of Vision Transformers for a Lightweight Skeleton-Based Action Recognition System
• What and Where: Modeling Skeletons from Semantic and Spatial Perspectives for Action Recognition
• Improving Skeleton-based Action Recognitionwith Robust Spatial and Temporal Features
• DeepActsNet: Spatial and Motion features from Face, Hands, and Body Combined with Convolutional and Graph Networks for Improved Action Recognition
• Hierarchical Action Classification with Network Pruning