Self-supervised transfer learning of physiological representations from free-living wearable data

Abstract

Wearable devices such as smartwatches are becoming increasingly popular tools for objectively monitoring physical activity in free-living conditions. To date, research has primarily focused on the purely supervised task of human activity recognition, demonstrating limited success in inferring high-level health outcomes from low-level signals. Here, we present a novel self-supervised representation learning method using activity and heart rate (HR) signals without semantic labels. With a deep neural network, we set HR responses as the supervisory signal for the activity data, leveraging their underlying physiological relationship. In addition, we propose a custom quantile loss function that accounts for the long-tailed HR distribution present in the general population. We evaluate our model in the largest free-living combined-sensing dataset (comprising >280k hours of wrist accelerometer & wearable ECG data). Our contributions are two-fold: i) the pre-training task creates a model that can accurately forecast HR based only on cheap activity sensors, and ii) we leverage the information captured through this task by proposing a simple method to aggregate the learnt latent representations (embeddings) from the window-level to user-level. Notably, we show that the embeddings can generalize in various downstream tasks through transfer learning with linear classifiers, capturing physiologically meaningful, personalized information. For instance, they can be used to predict variables associated with individuals' health, fitness and demographic characteristics, outperforming unsupervised autoencoders and common bio-markers. Overall, we propose the first multimodal self-supervised method for behavioral and physiological data with implications for large-scale health and lifestyle monitoring.

arXiv comment9 pages, 3 figures (long version of extended abstract arXiv:2011.04601)

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• XGBoost: A Scalable Tree Boosting System
• Distilling the Knowledge in a Neural Network
• Distributed Representations of Sentences and Documents
• Multi-task Self-Supervised Learning for Human Activity Detection
• Distributional Reinforcement Learning with Quantile Regression
• Self-supervised Learning for ECG-based Emotion Recognition
• Deep Activity Recognition Models with Triaxial Accelerometers
• DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
• Machine Learning for Scent: Learning Generalizable Perceptual Representations of Small Molecules
• DeepHeart: Semi-Supervised Sequence Learning for Cardiovascular Risk Prediction
• Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images
• Beyond expectation: Deep joint mean and quantile regression for spatio-temporal problems
• Online Heart Rate Prediction using Acceleration from a Wrist Worn Wearable
• Representation Learning on Variable Length and Incomplete Wearable-Sensory Time Series
• Learning Individualized Cardiovascular Responses from Large-scale Wearable Sensors Data

Cited by in corpus (5)

• SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
• From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques
• Longitudinal cardio-respiratory fitness prediction through wearables in free-living environments
• Self-supervised learning-based general laboratory progress pretrained model for cardiovascular event detection
• Uncovering Bias in Personal Informatics