Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition

Abstract

Automatic affective recognition has been an important research topic in human computer interaction (HCI) area. With recent development of deep learning techniques and large scale in-the-wild annotated datasets, the facial emotion analysis is now aimed at challenges in the real world settings. In this paper, we introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW2) Competition. In dealing with different emotion representations, including Categorical Emotions (CE), Action Units (AU), and Valence Arousal (VA), we propose a multi-task streaming network by a heuristic that the three representations are intrinsically associated with each other. Besides, we leverage an advanced facial expression embedding as prior knowledge, which is capable of capturing identity-invariant expression features while preserving the expression similarities, to aid the down-streaming recognition tasks. The extensive quantitative evaluations as well as ablation studies on the Aff-Wild2 dataset prove the effectiveness of our proposed prior aided streaming network approach.

References in corpus (8)

• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond
• Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework
• Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace
• Affect Expression Behaviour Analysis in the Wild using Spatio-Channel Attention and Complementary Context Information
• Action Units Recognition by Pairwise Deep Architecture

Cited by in corpus (6)

• Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition
• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations
• A Multi-task Mean Teacher for Semi-supervised Facial Affective Behavior Analysis
• Continuous Emotion Recognition with Audio-visual Leader-follower Attentive Fusion
• An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wild