Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition

Abstract

This paper contributes to the challenge of skeleton-based human action recognition in videos. The key step is to develop a generic network architecture to extract discriminative features for the spatio-temporal skeleton data. In this paper, we propose a novel module, namely Logsig-RNN, which is the combination of the log-signature layer and recurrent type neural networks (RNNs). The former one comes from the mathematically principled technology of signatures and log-signatures as representations for streamed data, which can manage high sample rate streams, non-uniform sampling and time series of variable length. It serves as an enhancement of the recurrent layer, which can be conveniently plugged into neural networks. Besides we propose two path transformation layers to significantly reduce path dimension while retaining the essential information fed into the Logsig-RNN module. Finally, numerical results demonstrate that replacing the RNN module by the Logsig-RNN module in SOTA networks consistently improves the performance on both Chalearn gesture data and NTU RGB+D 120 action data in terms of accuracy and robustness. In particular, we achieve the state-of-the-art accuracy on Chalearn2013 gesture data by combining simple path transformation layers with the Logsig-RNN. Codes are available at https://github.com/steveliao93/GCN_LogsigRNN.

arXiv commentThis paper is accepted by British Machine Vision Conference 2021

References in corpus (13)

• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition
• Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognition
• SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition
• Sparse arrays of signatures for online character recognition
• A Survey on 3D Skeleton-Based Action Recognition Using Learning Method
• Learning from the past, predicting the statistics for the future, learning an evolving system
• Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
• Extracting information from the signature of a financial data stream
• The iisignature library: efficient calculation of iterated-integral signatures and log signatures
• Rotation invariants of two dimensional curves based on iterated integrals
• Kernels for sequentially ordered data
• A multi-dimensional stream and its signature representation