SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation

Abstract

In recent years, vision transformers have been introduced into face recognition and analysis and have achieved performance breakthroughs. However, most previous methods generally train a single model or an ensemble of models to perform the desired task, which ignores the synergy among different tasks and fails to achieve improved prediction accuracy, increased data efficiency, and reduced training time. This paper presents a multi-purpose algorithm for simultaneous face recognition, facial expression recognition, age estimation, and face attribute estimation (40 attributes including gender) based on a single Swin Transformer. Our design, the SwinFace, consists of a single shared backbone together with a subnet for each set of related tasks. To address the conflicts among multiple tasks and meet the different demands of tasks, a Multi-Level Channel Attention (MLCA) module is integrated into each task-specific analysis subnet, which can adaptively select the features from optimal levels and channels to perform the desired tasks. Extensive experiments show that the proposed model has a better understanding of the face and achieves excellent performance for all tasks. Especially, it achieves 90.97% accuracy on RAF-DB and 0.22 $\epsilon$-error on CLAP2015, which are state-of-the-art results on facial expression recognition and age estimation respectively. The code and models will be made publicly available at https://github.com/lxq1000/SwinFace.

References in corpus (14)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Decoupled Weight Decay Regularization
• Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Learning Face Representation from Scratch
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments
• MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes
• Multi-task head pose estimation in-the-wild
• Consistent 3D Hand Reconstruction in Video via self-supervised Learning
• Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusion
• Face Transformer for Recognition
• Segment-based Methods for Facial Attribute Detection from Partial Faces

Cited by in corpus (5)

• Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey
• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning
• Unified Face Matching and Physical-Digital Spoofing Attack Detection
• Open-Set Biometrics: Beyond Good Closed-Set Models
• Overcoming Occlusions in the Wild: A Multi-Task Age Head Approach to Age Estimation