Continuous Human Action Recognition for Human-Machine Interaction: A Review

Abstract

With advances in data-driven machine learning research, a wide variety of prediction models have been proposed to capture spatio-temporal features for the analysis of video streams. Recognising actions and detecting action transitions within an input video are challenging but necessary tasks for applications that require real-time human-machine interaction. By reviewing a large body of recent related work in the literature, we thoroughly analyse, explain and compare action segmentation methods and provide details on the feature extraction and learning strategies that are used on most state-of-the-art methods. We cover the impact of the performance of object detection and tracking techniques on human action segmentation methodologies. We investigate the application of such models to real-world scenarios and discuss several limitations and key research directions towards improving interpretability, generalisation, optimisation and deployment.

arXiv commentPreprint submitted to ACM Computing Surveys

References in corpus (30)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SSD: Single Shot MultiBox Detector
• YOLOv4: Optimal Speed and Accuracy of Object Detection
• A Comprehensive Survey on Graph Neural Networks
• Conditional Generative Adversarial Nets
• LSTM: A Search Space Odyssey
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Simple Online and Realtime Tracking
• WaveNet: A Generative Model for Raw Audio
• YOLOX: Exceeding YOLO Series in 2021
• The Kinetics Human Action Video Dataset
• Striving for Simplicity: The All Convolutional Net
• Deep Visual Domain Adaptation: A Survey
• DSSD : Deconvolutional Single Shot Detector
• FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking
• A Survey of Deep Learning-based Object Detection
• Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
• Tracking without bells and whistles
• Captum: A unified and generic model interpretability library for PyTorch
• The THUMOS Challenge on Action Recognition for Videos "in the Wild"
• CSPNet: A New Backbone that can Enhance Learning Capability of CNN
• CornerNet-Lite: Efficient Keypoint Based Object Detection
• Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data
• Human Action Recognition and Prediction: A Survey
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• ByteTrack: Multi-Object Tracking by Associating Every Detection Box
• How Important Is a Neuron?
• Multi-modal Fusion for Single-Stage Continuous Gesture Recognition
• Domain-Specific Priors and Meta Learning for Few-Shot First-Person Action Recognition
• With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition