Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training

Abstract

Facial expression recognition (FER) in the wild is crucial for building reliable human-computer interactive systems. However, annotations of large scale datasets in FER has been a key challenge as these datasets suffer from noise due to various factors like crowd sourcing, subjectivity of annotators, poor quality of images, automatic labelling based on key word search etc. Such noisy annotations impede the performance of FER due to the memorization ability of deep networks. During early learning stage, deep networks fit on clean data. Then, eventually, they start overfitting on noisy labels due to their memorization ability, which limits FER performance. This report presents Consensual Collaborative Training (CCT) framework used in our submission to expression recognition track of the Affective Behaviour Analysis in-the-wild (ABAW) 2021 competition. CCT co-trains three networks jointly using a convex combination of supervision loss and consistency loss, without making any assumption about the noise distribution. A dynamic transition mechanism is used to move from supervision loss in early learning to consistency loss for consensus of predictions among networks in the later stage. Co-training reduces overall error, and consistency loss prevents overfitting to noisy samples. The performance of the model is validated on challenging Aff-Wild2 dataset for categorical expression classification. Our code is made publicly available at https://github.com/1980x/ABAW2021DMACS.

7 pages, 2 figures. arXiv admin note: substantial text overlap with arXiv:2009.14440. substantial text overlap with arXiv:2107.04746

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• A Closer Look at Memorization in Deep Networks
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition
• A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition
• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations
• Feature Pyramid Network for Multi-task Affective Analysis
• Multi-modal Affect Analysis using standardized data within subjects in the Wild

Cited by in corpus (2)

• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations