Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition

Abstract

EEG signals have been reported to be informative and reliable for emotion recognition in recent years. However, the inter-subject variability of emotion-related EEG signals still poses a great challenge for the practical applications of EEG-based emotion recognition. Inspired by recent neuroscience studies on inter-subject correlation, we proposed a Contrastive Learning method for Inter-Subject Alignment (CLISA) to tackle the cross-subject emotion recognition problem. Contrastive learning was employed to minimize the inter-subject differences by maximizing the similarity in EEG signal representations across subjects when they received the same emotional stimuli in contrast to different ones. Specifically, a convolutional neural network was applied to learn inter-subject aligned spatiotemporal representations from EEG time series in contrastive learning. The aligned representations were subsequently used to extract differential entropy features for emotion classification. CLISA achieved state-of-the-art cross-subject emotion recognition performance on our THU-EP dataset with 80 subjects and the publicly available SEED dataset with 15 subjects. It could generalize to unseen subjects or unseen emotional stimuli in testing. Furthermore, the spatiotemporal representations learned by CLISA could provide insights into the neural mechanisms of human emotion processing.

arXiv comment23 pages, 13 figures, journal paper. IEEE Transactions on Affective Computing, 2022

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
• Axiomatic Attribution for Deep Networks
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Transfer Learning in Brain-Computer Interfaces
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Pre-training of Graph Neural Network for Modeling Effects of Mutations on Protein-Protein Binding Affinity
• Correlated Components Analysis - Extracting Reliable Dimensions in Multivariate Data

Cited by in corpus (9)

• EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition
• Enhanced Cross-Dataset Electroencephalogram-based Emotion Recognition using Unsupervised Domain Adaptation
• Supervised Contrastive Learning for Affect Modelling
• Beware of Overestimated Decoding Performance Arising from Temporal Autocorrelations in Electroencephalogram Signals
• CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion
• Potential Indicator for Continuous Emotion Arousal by Dynamic Neural Synchrony
• Contrastive Learning for Sleep Staging based on Inter Subject Correlation
• Atoms of Thought: Universal EEG Representation Learning with Microstates
• Masked Generative-Contrastive Representation Learning for Cross-Dataset EEG-Based Emotion Recognition