LOTR: Face Landmark Localization Using Localization Transformer

Abstract

This paper presents a novel Transformer-based facial landmark localization network named Localization Transformer (LOTR). The proposed framework is a direct coordinate regression approach leveraging a Transformer network to better utilize the spatial information in the feature map. An LOTR model consists of three main modules: 1) a visual backbone that converts an input image into a feature map, 2) a Transformer module that improves the feature representation from the visual backbone, and 3) a landmark prediction head that directly predicts the landmark coordinates from the Transformer's representation. Given cropped-and-aligned face images, the proposed LOTR can be trained end-to-end without requiring any post-processing steps. This paper also introduces the smooth-Wing loss function, which addresses the gradient discontinuity of the Wing loss, leading to better convergence than standard loss functions such as L1, L2, and Wing loss. Experimental results on the JD landmark dataset provided by the First Grand Challenge of 106-Point Facial Landmark Localization indicate the superiority of LOTR over the existing methods on the leaderboard and two recent heatmap-based approaches. On the WFLW dataset, the proposed LOTR framework demonstrates promising results compared with several state-of-the-art methods. Additionally, we report the improvement in state-of-the-art face recognition performance when using our proposed LOTRs for face alignment.

arXiv commentAccepted for publication in IEEE Access

References in corpus (19)

• FaceNet: A Unified Embedding for Face Recognition and Clustering
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• High-Resolution Representations for Labeling Pixels and Regions
• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• RetinaFace: Single-stage Dense Face Localisation in the Wild
• SphereFace: Deep Hypersphere Embedding for Face Recognition
• Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments
• CosFace: Large Margin Cosine Loss for Deep Face Recognition
• GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
• Style Aggregated Network for Facial Landmark Detection
• Partial FC: Training 10 Million Identities on a Single Machine
• Facial Landmarks Localization using Cascaded Neural Networks
• Exploiting Facial Landmarks for Emotion Recognition in the Wild
• HIH: Towards More Accurate Face Alignment via Heatmap in Heatmap
• Face Detection with Feature Pyramids and Landmarks
• Grand Challenge of 106-Point Facial Landmark Localization
• Facial Action Unit Detection using 3D Facial Landmarks
• Sub-pixel face landmarks using heatmaps and a bag of tricks