Self-supervised Learning for ECG-based Emotion Recognition

Abstract

We present an electrocardiogram (ECG) -based emotion recognition system using self-supervised learning. Our proposed architecture consists of two main networks, a signal transformation recognition network and an emotion recognition network. First, unlabelled data are used to successfully train the former network to detect specific pre-determined signal transformations in the self-supervised learning step. Next, the weights of the convolutional layers of this network are transferred to the emotion recognition network, and two dense layers are trained in order to classify arousal and valence scores. We show that our self-supervised approach helps the model learn the ECG feature manifold required for emotion recognition, performing equal or better than the fully-supervised version of the model. Our proposed method outperforms the state-of-the-art in ECG-based emotion recognition with two publicly available datasets, SWELL and AMIGOS. Further analysis highlights the advantage of our self-supervised approach in requiring significantly less data to achieve acceptable results.

arXiv commentAccepted, 45th IEEE International Conference on Acoustics, Speech, and Signal Processing

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Multi-task Self-Supervised Learning for Human Activity Detection
• A Bayesian Deep Learning Framework for End-To-End Prediction of Emotion from Heartbeat
• Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning
• Self-supervised audio representation learning for mobile devices

Cited by in corpus (17)

• Self-Supervised Representation Learning: Introduction, Advances and Challenges
• Self-supervised ECG Representation Learning for Emotion Recognition
• Deep Learning for Time Series Forecasting: A Survey
• Subject-Aware Contrastive Learning for Biosignals
• A Transformer Architecture for Stress Detection from ECG
• Detection of Maternal and Fetal Stress from the Electrocardiogram with Self-Supervised Representation Learning
• Self-supervised transfer learning of physiological representations from free-living wearable data
• Self-supervised Contrastive Learning of Multi-view Facial Expressions
• In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection
• Self-Supervised Time Series Representation Learning by Inter-Intra Relational Reasoning
• Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion
• Coherent, super resolved radar beamforming using self-supervised learning
• Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction
• Self-Supervised Learning via multi-Transformation Classification for Action Recognition
• Learning Generalizable Physiological Representations from Large-scale Wearable Data
• Self-supervised Contrastive Learning for EEG-based Sleep Staging
• Consistency Training of Multi-exit Architectures for Sensor Data