Ensembles of Deep LSTM Learners for Activity Recognition using Wearables

Abstract

Recently, deep learning (DL) methods have been introduced very successfully into human activity recognition (HAR) scenarios in ubiquitous and wearable computing. Especially the prospect of overcoming the need for manual feature design combined with superior classification capabilities render deep neural networks very attractive for real-life HAR application. Even though DL-based approaches now outperform the state-of-the-art in a number of recognitions tasks of the field, yet substantial challenges remain. Most prominently, issues with real-life datasets, typically including imbalanced datasets and problematic data quality, still limit the effectiveness of activity recognition using wearables. In this paper we tackle such challenges through Ensembles of deep Long Short Term Memory (LSTM) networks. We have developed modified training procedures for LSTM networks and combine sets of diverse LSTM learners into classifier collectives. We demonstrate, both formally and empirically, that Ensembles of deep LSTM learners outperform the individual LSTM networks. Through an extensive experimental evaluation on three standard benchmarks (Opportunity, PAMAP2, Skoda) we demonstrate the excellent recognition capabilities of our approach and its potential for real-life applications of human activity recognition.

accepted for publication in ACM IMWUT (Ubicomp) 2017

Cited by in corpus (43)

• Deep Learning for Sensor-based Activity Recognition: A Survey
• Human Activity Recognition using Inertial, Physiological and Environmental Sensors: a Comprehensive Survey
• Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
• Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks
• SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
• Edge Intelligence: Architectures, Challenges, and Applications
• Exploring Cybersecurity Issues in 5G Enabled Electric Vehicle Charging Station with Deep Learning
• Attention-Based Deep Learning Framework for Human Activity Recognition with User Adaptation
• Improving Deep Learning for HAR with shallow LSTMs
• A Novel Multi-Stage Training Approach for Human Activity Recognition from Multimodal Wearable Sensor Data Using Deep Neural Network
• Semi-supervised Federated Learning for Activity Recognition
• Unsupervised Domain Adaptation for RF-based Gesture Recognition
• Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition
• RNNs on Monitoring Physical Activity Energy Expenditure in Older People
• Human activity recognition based on time series analysis using U-Net
• Fatigue Assessment using ECG and Actigraphy Sensors
• Multi-modal Egocentric Activity Recognition using Audio-Visual Features
• ConvBoost: Boosting ConvNets for Sensor-based Activity Recognition
• ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the Body
• Ubi-SleepNet: Advanced Multimodal Fusion Techniques for Three-stage Sleep Classification Using Ubiquitous Sensing
• Chronic-Pain Protective Behavior Detection with Deep Learning
• Wearable-based behaviour interpolation for semi-supervised human activity recognition
• Automatic Detection of Reflective Thinking in Mathematical Problem Solving based on Unconstrained Bodily Exploration
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• HAR-GCNN: Deep Graph CNNs for Human Activity Recognition From Highly Unlabeled Mobile Sensor Data
• RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
• IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition
• Spectrum-Guided Adversarial Disparity Learning
• Order Matters: Shuffling Sequence Generation for Video Prediction
• SOFA-Net: Second-Order and First-order Attention Network for Crowd Counting
• Adversary Helps: Gradient-based Device-Free Domain-Independent Gesture Recognition
• Adversarial Transferability in Wearable Sensor Systems
• Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data
• Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensors
• T-WaveNet: Tree-Structured Wavelet Neural Network for Sensor-Based Time Series Analysis
• Fullie and Wiselie: A Dual-Stream Recurrent Convolutional Attention Model for Activity Recognition
• MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition with Multi-Domain Deep Learning Model
• Hierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition
• Ballroom Dance Movement Recognition Using a Smart Watch
• Multi-agent Attentional Activity Recognition
• Are Accelerometers for Activity Recognition a Dead-end?
• Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention
• Towards Reliable, Automated General Movement Assessment for Perinatal Stroke Screening in Infants Using Wearable Accelerometers