Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition

Abstract

As Facial Expression Recognition (FER) systems become integrated into our daily lives, these systems need to prioritise making fair decisions instead of aiming at higher individual accuracy scores. Ranging from surveillance systems to diagnosing mental and emotional health conditions of individuals, these systems need to balance the accuracy vs fairness trade-off to make decisions that do not unjustly discriminate against specific under-represented demographic groups. Identifying bias as a critical problem in facial analysis systems, different methods have been proposed that aim to mitigate bias both at data and algorithmic levels. In this work, we propose the novel usage of Continual Learning (CL), in particular, using Domain-Incremental Learning (Domain-IL) settings, as a potent bias mitigation method to enhance the fairness of FER systems while guarding against biases arising from skewed data distributions. We compare different non-CL-based and CL-based methods for their classification accuracy and fairness scores on expression recognition and Action Unit (AU) detection tasks using two popular benchmarks, the RAF-DB and BP4D datasets, respectively. Our experimental results show that CL-based methods, on average, outperform other popular bias mitigation techniques on both accuracy and fairness metrics.

References in corpus (20)

• Overcoming catastrophic forgetting in neural networks
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Equality of Opportunity in Supervised Learning
• Deep Facial Expression Recognition: A Survey
• Continual Learning Through Synaptic Intelligence
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Three scenarios for continual learning
• FearNet: Brain-Inspired Model for Incremental Learning
• Progress & Compress: A scalable framework for continual learning
• Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Measuring Catastrophic Forgetting in Neural Networks
• Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
• A Deeper Look at Facial Expression Dataset Bias
• Deep Generative Dual Memory Network for Continual Learning
• Don't forget, there is more than forgetting: new metrics for Continual Learning
• Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization
• Functional Regularisation for Continual Learning with Gaussian Processes
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• Spatio-Temporal Analysis of Facial Actions using Lifecycle-Aware Capsule Networks