Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations

Abstract

Presence of noise in the labels of large scale facial expression datasets has been a key challenge towards Facial Expression Recognition (FER) in the wild. During early learning stage, deep networks fit on clean data. Then, eventually, they start overfitting on noisy labels due to their memorization ability, which limits FER performance. This work proposes an effective training strategy in the presence of noisy labels, called as Consensual Collaborative Training (CCT) framework. CCT co-trains three networks jointly using a convex combination of supervision loss and consistency loss, without making any assumption about the noise distribution. A dynamic transition mechanism is used to move from supervision loss in early learning to consistency loss for consensus of predictions among networks in the later stage. Inference is done using a single network based on a simple knowledge distillation scheme. Effectiveness of the proposed framework is demonstrated on synthetic as well as real noisy FER datasets. In addition, a large test subset of around 5K images is annotated from the FEC dataset using crowd wisdom of 16 different annotators and reliable labels are inferred. CCT is also validated on it. State-of-the-art performance is reported on the benchmark FER datasets RAFDB (90.84%) FERPlus (89.99%) and AffectNet (66%). Our codes are available at https://github.com/1980x/CCT.

arXiv comment11 pages, 6 figures, Published with International Journal of Engineering Trends and Technology (IJETT), Codes: https://github.com/1980x/CCT

References in corpus (22)

• Distilling the Knowledge in a Neural Network
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Deep Facial Expression Recognition: A Survey
• Learning to Reweight Examples for Robust Deep Learning
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond
• A Closer Look at Memorization in Deep Networks
• Local Learning with Deep and Handcrafted Features for Facial Expression Recognition
• Decoupling "when to update" from "how to update"
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
• Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition
• A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition
• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations
• Feature Pyramid Network for Multi-task Affective Analysis
• Multi-modal Affect Analysis using standardized data within subjects in the Wild
• THIN: THrowable Information Networks and Application for Facial Expression Recognition In The Wild

Cited by in corpus (2)

• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations