Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition

Abstract

EEG signals have been reported to be informative and reliable for emotion recognition in recent years. However, the inter-subject variability of emotion-related EEG signals still poses a great challenge for the practical applications of EEG-based emotion recognition. Inspired by recent neuroscience studies on inter-subject correlation, we proposed a Contrastive Learning method for Inter-Subject Alignment (CLISA) to tackle the cross-subject emotion recognition problem. Contrastive learning was employed to minimize the inter-subject differences by maximizing the similarity in EEG signal representations across subjects when they received the same emotional stimuli in contrast to different ones. Specifically, a convolutional neural network was applied to learn inter-subject aligned spatiotemporal representations from EEG time series in contrastive learning. The aligned representations were subsequently used to extract differential entropy features for emotion classification. CLISA achieved state-of-the-art cross-subject emotion recognition performance on our THU-EP dataset with 80 subjects and the publicly available SEED dataset with 15 subjects. It could generalize to unseen subjects or unseen emotional stimuli in testing. Furthermore, the spatiotemporal representations learned by CLISA could provide insights into the neural mechanisms of human emotion processing.

23 pages, 13 figures, journal paper. IEEE Transactions on Affective Computing, 2022

References in corpus (1)

• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Cited by in corpus (9)

• EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition
• Enhanced Cross-Dataset Electroencephalogram-based Emotion Recognition using Unsupervised Domain Adaptation
• Supervised Contrastive Learning for Affect Modelling
• Beware of Overestimated Decoding Performance Arising from Temporal Autocorrelations in Electroencephalogram Signals
• CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion
• Potential Indicator for Continuous Emotion Arousal by Dynamic Neural Synchrony
• Contrastive Learning for Sleep Staging based on Inter Subject Correlation
• Atoms of Thought: Universal EEG Representation Learning with Microstates
• Masked Generative-Contrastive Representation Learning for Cross-Dataset EEG-Based Emotion Recognition