EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition

Abstract

Speech emotion recognition (SER) systems often exhibit gender bias. However, the effectiveness and robustness of existing debiasing methods in such multi-label scenarios remain underexplored. To address this gap, we present EMO-Debias, a large-scale comparison of 13 debiasing methods applied to multi-label SER. Our study encompasses techniques from pre-processing, regularization, adversarial learning, biased learners, and distributionally robust optimization. Experiments conducted on acted and naturalistic emotion datasets, using WavLM and XLSR representations, evaluate each method under conditions of gender imbalance. Our analysis quantifies the trade-offs between fairness and accuracy, identifying which approaches consistently reduce gender performance gaps without compromising overall model performance. The findings provide actionable insights for selecting effective debiasing strategies and highlight the impact of dataset distributions.

8 pages

References in corpus (6)

• User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition in Federated Learning
• Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition
• Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
• On the social bias of speech self-supervised models
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models
• Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approach