SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data

Abstract

Machine learning and deep learning have shown great promise in mobile sensing applications, including Human Activity Recognition. However, the performance of such models in real-world settings largely depends on the availability of large datasets that captures diverse behaviors. Recently, studies in computer vision and natural language processing have shown that leveraging massive amounts of unlabeled data enables performance on par with state-of-the-art supervised models. In this work, we present SelfHAR, a semi-supervised model that effectively learns to leverage unlabeled mobile sensing datasets to complement small labeled datasets. Our approach combines teacher-student self-training, which distills the knowledge of unlabeled and labeled datasets while allowing for data augmentation, and multi-task self-supervision, which learns robust signal-level representations by predicting distorted versions of the input. We evaluated SelfHAR on various HAR datasets and showed state-of-the-art performance over supervised and previous semi-supervised approaches, with up to 12% increase in F1 score using the same number of model parameters at inference. Furthermore, SelfHAR is data-efficient, reaching similar performance using up to 10 times less labeled data compared to supervised approaches. Our work not only achieves state-of-the-art performance in a diverse set of HAR datasets, but also sheds light on how pre-training tasks may affect downstream performance.

Accepted for publication in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2021

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• How transferable are features in deep neural networks?
• Semi-Supervised Learning with Deep Generative Models
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Learning Representations by Maximizing Mutual Information Across Views
• Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring using Convolutional Neural Networks
• Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
• Self-supervised ECG Representation Learning for Emotion Recognition
• Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables
• Multi-task Self-Supervised Learning for Human Activity Detection
• Billion-scale semi-supervised learning for image classification
• Protecting Sensory Data against Sensitive Inferences
• Towards Using Unlabeled Data in a Sparse-coding Framework for Human Activity Recognition
• Self-supervised transfer learning of physiological representations from free-living wearable data

Cited by in corpus (12)

• Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data
• COCOA: Cross Modality Contrastive Learning for Sensor Data
• Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion
• ConvBoost: Boosting ConvNets for Sensor-based Activity Recognition
• Self-supervised learning-based general laboratory progress pretrained model for cardiovascular event detection
• DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing
• PRIMUS: Pretraining IMU Encoders with Multimodal Self-Supervision
• Evaluation of self-supervised pre-training for automatic infant movement classification using wearable movement sensors
• Guided-GAN: Adversarial Representation Learning for Activity Recognition with Wearables
• Transformer-Based Behavioral Representation Learning Enables Transfer Learning for Mobile Sensing in Small Datasets
• Experience Paper: Adopting Activity Recognition in On-demand Food Delivery Business
• Consistency Training of Multi-exit Architectures for Sensor Data