Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities

Abstract

The vast proliferation of sensor devices and Internet of Things enables the applications of sensor-based activity recognition. However, there exist substantial challenges that could influence the performance of the recognition system in practical scenarios. Recently, as deep learning has demonstrated its effectiveness in many areas, plenty of deep methods have been investigated to address the challenges in activity recognition. In this study, we present a survey of the state-of-the-art deep learning methods for sensor-based human activity recognition. We first introduce the multi-modality of the sensory data and provide information for public datasets that can be used for evaluation in different challenge tasks. We then propose a new taxonomy to structure the deep methods by challenges. Challenges and challenge-related deep methods are summarized and analyzed to form an overview of the current research progress. At the end of this work, we discuss the open issues and provide some insights for future directions.

References in corpus (18)

• LSTM: A Search Space Odyssey
• Deep Learning for Sensor-based Activity Recognition: A Survey
• Visualizing and Understanding Recurrent Networks
• Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
• Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables
• Mobile Sensor Data Anonymization
• Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks
• Deep Activity Recognition Models with Triaxial Accelerometers
• Language Generation with Recurrent Generative Adversarial Networks without Pre-training
• Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art
• Detecting Unseen Falls from Wearable Devices using Channel-wise Ensemble of Autoencoders
• Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface
• DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition
• Concurrent Activity Recognition with Multimodal CNN-LSTM Structure
• Human activity recognition based on time series analysis using U-Net
• An Interpretable Machine Vision Approach to Human Activity Recognition using Photoplethysmograph Sensor Data
• ActiveHARNet: Towards On-Device Deep Bayesian Active Learning for Human Activity Recognition
• Personalized Human Activity Recognition Using Convolutional Neural Networks

Cited by in corpus (13)

• A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review
• Hang-Time HAR: A Benchmark Dataset for Basketball Activity Recognition using Wrist-Worn Inertial Sensors
• Deep Generative Domain Adaptation with Temporal Attention for Cross-User Activity Recognition
• Spectrum-Guided Adversarial Disparity Learning
• Comparison of Deep Learning Techniques on Human Activity Recognition using Ankle Inertial Signals
• Real-Time Activity Recognition and Intention Recognition Using a Vision-based Embedded System
• Video-based Pose-Estimation Data as Source for Transfer Learning in Human Activity Recognition
• Early Mobility Recognition for Intensive Care Unit Patients Using Accelerometers
• T-WaveNet: Tree-Structured Wavelet Neural Network for Sensor-Based Time Series Analysis
• Entropy Decision Fusion for Smartphone Sensor based Human Activity Recognition
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification