Improving Skeleton-based Action Recognitionwith Robust Spatial and Temporal Features

Abstract

Recently skeleton-based action recognition has made signif-icant progresses in the computer vision community. Most state-of-the-art algorithms are based on Graph Convolutional Networks (GCN), andtarget at improving the network structure of the backbone GCN lay-ers. In this paper, we propose a novel mechanism to learn more robustdiscriminative features in space and time. More specifically, we add aDiscriminative Feature Learning (DFL) branch to the last layers of thenetwork to extract discriminative spatial and temporal features to helpregularize the learning. We also formally advocate the use of Direction-Invariant Features (DIF) as input to the neural networks. We show thataction recognition accuracy can be improved when these robust featuresare learned and used. We compare our results with those of ST-GCNand related methods on four datasets: NTU-RGBD60, NTU-RGBD120,SYSU 3DHOI and Skeleton-Kinetics.

References in corpus (6)

• The Kinetics Human Action Video Dataset
• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
• AlignedReID: Surpassing Human-Level Performance in Person Re-Identification
• Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition