Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospects

Abstract

Human Activity Recognition (HAR) plays a significant role in the everyday life of people because of its ability to learn extensive high-level information about human activity from wearable or stationary devices. A substantial amount of research has been conducted on HAR and numerous approaches based on deep learning and machine learning have been exploited by the research community to classify human activities. The main goal of this review is to summarize recent works based on a wide range of deep neural networks architecture, namely convolutional neural networks (CNNs) for human activity recognition. The reviewed systems are clustered into four categories depending on the use of input devices like multimodal sensing devices, smartphones, radar, and vision devices. This review describes the performances, strengths, weaknesses, and the used hyperparameters of CNN architectures for each reviewed system with an overview of available public data sources. In addition, a discussion with the current challenges to CNN-based HAR systems is presented. Finally, this review is concluded with some potential future directions that would be of great assistance for the researchers who would like to contribute to this field.

32 pages, 4 figures, 4 Tables

References in corpus (9)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• YouTube-8M: A Large-Scale Video Classification Benchmark
• Attention-based Convolutional Neural Network for Weakly Labeled Human Activities Recognition with Wearable Sensors
• Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition
• PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding
• EmbraceNet: A robust deep learning architecture for multimodal classification
• A Data Driven End-to-end Approach for In-the-wild Monitoring of Eating Behavior Using Smartwatches
• A Novel Multi-Stage Training Approach for Human Activity Recognition from Multimodal Wearable Sensor Data Using Deep Neural Network

Cited by in corpus (1)

• A Lightweight 3D-CNN for Event-Based Human Action Recognition with Privacy-Preserving Potential