Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition

Abstract

Facial Expression Recognition (FER) in the wild is an extremely challenging task. Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural Networks (CNN). This is mainly because the new proposed modules are difficult to converge well from scratch due to lacking inductive bias and easy to focus on the occlusion and noisy areas. TransFER, a representative transformer-based method for FER, alleviates this with multi-branch attention dropping but brings excessive computations. On the contrary, we present two attentive pooling (AP) modules to pool noisy features directly. The AP modules include Attentive Patch Pooling (APP) and Attentive Token Pooling (ATP). They aim to guide the model to emphasize the most discriminative features while reducing the impacts of less relevant features. The proposed APP is employed to select the most informative patches on CNN features, and ATP discards unimportant tokens in ViT. Being simple to implement and without learnable parameters, the APP and ATP intuitively reduce the computational cost while boosting the performance by ONLY pursuing the most discriminative features. Qualitative results demonstrate the motivations and effectiveness of our attentive poolings. Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods.

Codes will be public on https://github.com/youqingxiaozhua/APViT

References in corpus (6)

• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• DeepViT: Towards Deeper Vision Transformer
• Expression Analysis Based on Face Regions in Read-world Conditions
• Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition
• A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition

Cited by in corpus (5)

• From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
• PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers
• AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels
• M2Former: Multi-Scale Patch Selection for Fine-Grained Visual Recognition