Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression

Abstract

In automatic emotion recognition (AER), labels assigned by different human annotators to the same utterance are often inconsistent due to the inherent complexity of emotion and the subjectivity of perception. Though deterministic labels generated by averaging or voting are often used as the ground truth, it ignores the intrinsic uncertainty revealed by the inconsistent labels. This paper proposes a Bayesian approach, deep evidential emotion regression (DEER), to estimate the uncertainty in emotion attributes. Treating the emotion attribute labels of an utterance as samples drawn from an unknown Gaussian distribution, DEER places an utterance-specific normal-inverse gamma prior over the Gaussian likelihood and predicts its hyper-parameters using a deep neural network model. It enables a joint estimation of emotion attributes along with the aleatoric and epistemic uncertainties. AER experiments on the widely used MSP-Podcast and IEMOCAP datasets showed DEER produced state-of-the-art results for both the mean values and the distribution of emotion attributes.

arXiv commentAccepted by ACL 2023

References in corpus (12)

• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• Evidential Deep Learning to Quantify Classification Uncertainty
• SpeechBrain: A General-Purpose Speech Toolkit
• Predictive Uncertainty Estimation via Prior Networks
• SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild
• Emotion recognition by fusing time synchronous and time asynchronous representations
• Two-stage dimensional emotion recognition by fusing predictions of acoustic and text networks using SVM
• Probing Speech Emotion Recognition Transformers for Linguistic Knowledge
• Estimating the Uncertainty in Emotion Class Labels with Utterance-Specific Dirichlet Priors
• End-To-End Label Uncertainty Modeling for Speech-based Arousal Recognition Using Bayesian Neural Networks