Think about boundary: Fusing multi-level boundary information for landmark heatmap regression

Abstract

Although current face alignment algorithms have obtained pretty good performances at predicting the location of facial landmarks, huge challenges remain for faces with severe occlusion and large pose variations, etc. On the contrary, semantic location of facial boundary is more likely to be reserved and estimated on these scenes. Therefore, we study a two-stage but end-to-end approach for exploring the relationship between the facial boundary and landmarks to get boundary-aware landmark predictions, which consists of two modules: the self-calibrated boundary estimation (SCBE) module and the boundary-aware landmark transform (BALT) module. In the SCBE module, we modify the stem layers and employ intermediate supervision to help generate high-quality facial boundary heatmaps. Boundary-aware features inherited from the SCBE module are integrated into the BALT module in a multi-scale fusion framework to better model the transformation from boundary to landmark heatmap. Experimental results conducted on the challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

12 pages, 9 figures

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• High-Resolution Representations for Labeling Pixels and Regions
• Stacked Hourglass Networks for Human Pose Estimation
• Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
• Occlusion Coherence: Detecting and Localizing Occluded Faces
• Style Aggregated Network for Facial Landmark Detection
• Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
• PFLD: A Practical Facial Landmark Detector
• Learning Dense Facial Correspondences in Unconstrained Images
• LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood
• Dynamic Attention-controlled Cascaded Shape Regression Exploiting Training Data Augmentation and Fuzzy-set Sample Weighting