Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training

Abstract

Facial expression recognition (FER) in the wild is crucial for building reliable human-computer interactive systems. However, annotations of large scale datasets in FER has been a key challenge as these datasets suffer from noise due to various factors like crowd sourcing, subjectivity of annotators, poor quality of images, automatic labelling based on key word search etc. Such noisy annotations impede the performance of FER due to the memorization ability of deep networks. During early learning stage, deep networks fit on clean data. Then, eventually, they start overfitting on noisy labels due to their memorization ability, which limits FER performance. This report presents Consensual Collaborative Training (CCT) framework used in our submission to expression recognition track of the Affective Behaviour Analysis in-the-wild (ABAW) 2021 competition. CCT co-trains three networks jointly using a convex combination of supervision loss and consistency loss, without making any assumption about the noise distribution. A dynamic transition mechanism is used to move from supervision loss in early learning to consistency loss for consensus of predictions among networks in the later stage. Co-training reduces overall error, and consistency loss prevents overfitting to noisy samples. The performance of the model is validated on challenging Aff-Wild2 dataset for categorical expression classification. Our code is made publicly available at https://github.com/1980x/ABAW2021DMACS.

arXiv comment7 pages, 2 figures. arXiv admin note: substantial text overlap with arXiv:2009.14440. substantial text overlap with arXiv:2107.04746

References in corpus (22)

• Distilling the Knowledge in a Neural Network
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Deep Facial Expression Recognition: A Survey
• Learning to Reweight Examples for Robust Deep Learning
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond
• A Closer Look at Memorization in Deep Networks
• Local Learning with Deep and Handcrafted Features for Facial Expression Recognition
• Decoupling "when to update" from "how to update"
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
• Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition
• A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition
• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations
• Feature Pyramid Network for Multi-task Affective Analysis
• Multi-modal Affect Analysis using standardized data within subjects in the Wild
• THIN: THrowable Information Networks and Application for Facial Expression Recognition In The Wild

Cited by in corpus (2)

• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations