Multi-task head pose estimation in-the-wild

Abstract

We present a deep learning-based multi-task approach for head pose estimation in images. We contribute with a network architecture and training strategy that harness the strong dependencies among face pose, alignment and visibility, to produce a top performing model for all three tasks. Our architecture is an encoder-decoder CNN with residual blocks and lateral skip connections. We show that the combination of head pose estimation and landmark-based face alignment significantly improve the performance of the former task. Further, the location of the pose task at the bottleneck layer, at the end of the encoder, and that of tasks depending on spatial information, such as visibility and alignment, in the final decoder layer, also contribute to increase the final performance. In the experiments conducted the proposed model outperforms the state-of-the-art in the face pose and visibility tasks. By including a final landmark regression step it also produces face alignment results on par with the state-of-the-art.

References in corpus (10)

• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• Facial Landmark Detection: a Literature Survey
• Face Alignment Across Large Poses: A 3D Solution
• Face Alignment in Full Pose Range: A 3D Total Solution
• Which Tasks Should Be Learned Together in Multi-task Learning?
• Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources
• Self-supervised learning of a facial attribute embedding from video
• Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection
• Face Alignment using a 3D Deeply-initialized Ensemble of Regression Trees

Cited by in corpus (4)

• SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation
• SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction
• On the representation and methodology for wide and short range head pose estimation
• Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data