Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition

Abstract

When recognizing emotions, subtle nuances in displays of emotion generate ambiguity or uncertainty in emotion perception. Emotion uncertainty has been previously interpreted as inter-rater disagreement among multiple annotators. In this paper, we consider a more common and challenging scenario: modeling emotion uncertainty when only single emotion labels are available. From a Bayesian perspective, we propose to use deep ensembles to capture uncertainty for multiple emotion descriptors, i.e., action units, discrete expression labels and continuous descriptors. We further apply iterative self-distillation. Iterative distillation over multiple generations significantly improves performance in both emotion recognition and uncertainty estimation. Our method generates single student models that provide accurate estimates of uncertainty for in-domain samples and a student ensemble that can detect out-of-domain samples. Our experiments on emotion recognition and uncertainty estimation using the Aff-wild2 dataset demonstrate that our algorithm gives more reliable uncertainty estimates than both Temperature Scaling and Monte Carol Dropout.

arXiv commentAccepted as a Workshop paper in ICCV2021 proceeding

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• When Does Label Smoothing Help?
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond
• Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
• Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
• Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework
• Self-Distillation Amplifies Regularization in Hilbert Space
• Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace