Deep Learning for Sensor-based Activity Recognition: A Survey

Abstract

Sensor-based activity recognition seeks the profound high-level knowledge about human activities from multitudes of low-level sensor readings. Conventional pattern recognition approaches have made tremendous progress in the past years. However, those methods often heavily rely on heuristic hand-crafted feature extraction, which could hinder their generalization performance. Additionally, existing methods are undermined for unsupervised and incremental learning tasks. Recently, the recent advancement of deep learning makes it possible to perform automatic high-level feature extraction thus achieves promising performance in many areas. Since then, deep learning based methods have been widely adopted for the sensor-based activity recognition tasks. This paper surveys the recent advance of deep learning based sensor-based activity recognition. We summarize existing literature from three aspects: sensor modality, deep model, and application. We also present detailed insights on existing work and propose grand challenges for future research.

10 pages, 2 figures, and 5 tables; submitted to Pattern Recognition Letters (second revision)

References in corpus (5)

• Deep Learning in Neural Networks: An Overview
• Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
• Sensor-based Gait Parameter Extraction with Deep Convolutional Neural Networks
• Deep Recurrent Neural Network for Mobile Human Activity Recognition with High Throughput
• A Deep Learning Framework using Passive WiFi Sensing for Respiration Monitoring

Cited by in corpus (94)

• Deep learning for time series classification: a review
• Human Activity Recognition using Inertial, Physiological and Environmental Sensors: a Comprehensive Survey
• Multi-task Self-Supervised Learning for Human Activity Detection
• Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
• Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospects
• A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning
• Different Approaches for Human Activity Recognition: A Survey
• TS-CHIEF: A Scalable and Accurate Forest Algorithm for Time Series Classification
• Time series forecasting of new cases and new deaths rate for COVID-19 using deep learning methods
• Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Layer-wise training convolutional neural networks with smaller filters for human activity recognition using wearable sensors
• Deep Learning in Mobile and Wireless Networking: A Survey
• Meta-HAR: Federated Representation Learning for Human Activity Recognition
• A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
• Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence
• LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation -- Extended Version
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition
• Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art
• Attention-Based Deep Learning Framework for Human Activity Recognition with User Adaptation
• An Overview of Human Activity Recognition Using Wearable Sensors: Healthcare and Artificial Intelligence
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• Deep Learning-based Cattle Activity Classification Using Joint Time-frequency Data Representation
• Cross-domain Activity Recognition via Substructural Optimal Transport
• Sequential Weakly Labeled Multi-Activity Localization and Recognition on Wearable Sensors using Recurrent Attention Networks
• Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review
• FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
• Sequential Decision Fusion for Environmental Classification in Assistive Walking
• Stratified Transfer Learning for Cross-domain Activity Recognition
• Exploring the Generalizability of Spatio-Temporal Traffic Prediction: Meta-Modeling and an Analytic Framework
• Accurate Discharge Coefficient Prediction of Streamlined Weirs by Coupling Linear Regression and Deep Convolutional Gated Recurrent Unit
• Video Action Understanding
• Smartphone Sensor-based Human Activity Recognition Robust to Different Sampling Rates
• Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables
• Human activity recognition based on time series analysis using U-Net
• Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals
• Balanced Distribution Adaptation for Transfer Learning
• Attributes' Importance for Zero-Shot Pose-Classification Based on Wearable Sensors
• Unveiling the frontiers of deep learning: innovations shaping diverse domains
• DANA: Dimension-Adaptive Neural Architecture for Multivariate Sensor Data
• Deep learning models for predictive maintenance: a survey, comparison, challenges and prospect
• A Multi-Modal Feature Embedding Approach to Diagnose Alzheimer Disease from Spoken Language
• CycleSense: Detecting Near Miss Incidents in Bicycle Traffic from Mobile Motion Sensors
• Wearable Sensor Data Based Human Activity Recognition using Machine Learning: A new approach
• DAPnet: A Double Self-attention Convolutional Network for Point Cloud Semantic Labeling
• An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition
• Gradient Amplification: An efficient way to train deep neural networks
• HAR-GCNN: Deep Graph CNNs for Human Activity Recognition From Highly Unlabeled Mobile Sensor Data
• Similarity Embedding Networks for Robust Human Activity Recognition
• ActiveHARNet: Towards On-Device Deep Bayesian Active Learning for Human Activity Recognition
• Clustered FedStack: Intermediate Global Models with Bayesian Information Criterion
• Transfer Learning for Human Activity Recognition using Representational Analysis of Neural Networks
• Learning Not to Spoof
• DeePLT: Personalized Lighting Facilitates by Trajectory Prediction of Recognized Residents in the Smart Home
• Deep learning for time series classification
• Topological Persistence Guided Knowledge Distillation for Wearable Sensor Data
• Benchmarking Conventional Vision Models on Neuromorphic Fall Detection and Action Recognition Dataset
• Joint Distribution and Transitions of Pain and Activity in Critically Ill Patients
• Fusion of Deep Neural Networks for Activity Recognition: A Regular Vine Copula Based Approach
• Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery
• MakeSense: An IoT Testbed for Social Research of Indoor Activities
• MoSen: Activity Modelling in Multiple-Occupancy Smart Homes
• Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches
• Deep Smartphone Sensors-WiFi Fusion for Indoor Positioning and Tracking
• LLT: An R package for Linear Law-based Feature Space Transformation
• A Domain-specific Language and Architecture for Detecting Process Activities from Sensor Streams in IoT
• Deep multi-modal networks for book genre classification based on its cover
• FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoring
• A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
• Latent hypernet: Exploring all Layers from Convolutional Neural Networks
• Few-Shot Learning-Based Human Activity Recognition
• Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensors
• A Tree-structure Convolutional Neural Network for Temporal Features Exaction on Sensor-based Multi-resident Activity Recognition
• A*HAR: A New Benchmark towards Semi-supervised learning for Class-imbalanced Human Activity Recognition
• Invariant Feature Learning for Sensor-based Human Activity Recognition
• T-WaveNet: Tree-Structured Wavelet Neural Network for Sensor-Based Time Series Analysis
• Spiking Two-Stream Methods with Unsupervised STDP-based Learning for Action Recognition
• A Survey of Challenges and Opportunities in Sensing and Analytics for Cardiovascular Disorders
• A Transfer Learning Method for Goal Recognition Exploiting Cross-Domain Spatial Features
• Rapid Identification of X-ray Diffraction Spectra Based on Very Limited Data by Interpretable Convolutional Neural Networks
• On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?
• Machine Learning Systems for Intelligent Services in the IoT: A Survey
• Am I fit for this physical activity? Neural embedding of physical conditioning from inertial sensors
• Combining Deep Learning Classifiers for 3D Action Recognition
• Simultaneous Learning from Human Pose and Object Cues for Real-Time Activity Recognition
• A comparative study on wearables and single-camera video for upper-limb out-of-thelab activity recognition with different deep learning architectures
• Octave Mix: Data augmentation using frequency decomposition for activity recognition
• Improving state estimation through projection post-processing for activity recognition with application to football
• Wheelchair Behavior Recognition for Visualizing Sidewalk Accessibility by Deep Neural Networks
• Incremental Learning Techniques for Online Human Activity Recognition
• Effective Human Activity Recognition Based on Small Datasets
• A systematic review of Python packages for time series analysis
• Are Accelerometers for Activity Recognition a Dead-end?