PARSE: Pairwise Alignment of Representations in Semi-Supervised EEG Learning for Emotion Recognition
arXiv:2202.05400 · doi:10.1109/TAFFC.2022.3210441
Abstract
We propose PARSE, a novel semi-supervised architecture for learning strong EEG representations for emotion recognition. To reduce the potential distribution mismatch between the large amounts of unlabeled data and the limited amount of labeled data, PARSE uses pairwise representation alignment. First, our model performs data augmentation followed by label guessing for large amounts of original and augmented unlabeled data. This is then followed by sharpening of the guessed labels and convex combinations of the unlabeled and labeled data. Finally, representation alignment and emotion classification are performed. To rigorously test our model, we compare PARSE to several state-of-the-art semi-supervised approaches which we implement and adapt for EEG learning. We perform these experiments on four public EEG-based emotion recognition datasets, SEED, SEED-IV, SEED-V and AMIGOS (valence and arousal). The experiments show that our proposed framework achieves the overall best results with varying amounts of limited labeled samples in SEED, SEED-IV and AMIGOS (valence), while approaching the overall best result (reaching the second-best) in SEED-V and AMIGOS (arousal). The analysis shows that our pairwise representation alignment considerably improves the performance by reducing the distribution alignment between unlabeled and labeled data, especially when only 1 sample per class is labeled.
arXiv commentAccepted in IEEE Transactions of Affective Computing
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- mixup: Beyond Empirical Risk Minimization
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Temporal Ensembling for Semi-Supervised Learning
- Interpolation Consistency Training for Semi-Supervised Learning
- Spatial-Temporal Recurrent Neural Network for Emotion Recognition
- Self-supervised ECG Representation Learning for Emotion Recognition
- Classification of Hand Movements from EEG using a Deep Attention-based LSTM Network
- Capsule Attention for Multimodal EEG-EOG Representation Learning with Application to Driver Vigilance Estimation
- Multimodal Emotion Recognition Using Deep Canonical Correlation Analysis
- On Data-Augmentation and Consistency-Based Semi-Supervised Learning
- Distilling EEG Representations via Capsules for Affective Computing