Facial Landmark Detection: a Literature Survey

Abstract

The locations of the fiducial facial landmark points around facial components and facial contour capture the rigid and non-rigid facial deformations due to head movements and facial expressions. They are hence important for various facial analysis tasks. Many facial landmark detection algorithms have been developed to automatically detect those key points over the years, and in this paper, we perform an extensive review of them. We classify the facial landmark detection algorithms into three major categories: holistic methods, Constrained Local Model (CLM) methods, and the regression-based methods. They differ in the ways to utilize the facial appearance and shape information. The holistic methods explicitly build models to represent the global facial appearance and shape information. The CLMs explicitly leverage the global shape model but build the local appearance models. The regression-based methods implicitly capture facial shape and appearance information. For algorithms within each category, we discuss their underlying theories as well as their differences. We also compare their performances on both controlled and in the wild benchmark datasets, under varying facial expressions, head poses, and occlusion. Based on the evaluations, we point out their respective strengths and weaknesses. There is also a separate section to review the latest deep learning-based algorithms. The survey also includes a listing of the benchmark databases and existing software. Finally, we identify future research directions, including combining methods in different categories to leverage their respective strengths to solve landmark detection "in-the-wild".

References in corpus (6)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• DeepID3: Face Recognition with Very Deep Neural Networks
• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild"
• Pose-Invariant 3D Face Alignment

Cited by in corpus (31)

• Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks
• Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion
• Multi-task head pose estimation in-the-wild
• Faster Training of Mask R-CNN by Focusing on Instance Boundaries
• Structured Landmark Detection via Topology-Adapting Deep Graph Learning
• Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction
• Cross-Modal Health State Estimation
• Automated Detection of Cat Facial Landmarks
• Non-contact Atrial Fibrillation Detection from Face Videos by Learning Systolic Peaks
• Automatic vocal tract landmark localization from midsagittal MRI data
• Neural Sign Language Translation based on Human Keypoint Estimation
• Unveiling the Human-like Similarities of Automatic Facial Expression Recognition: An Empirical Exploration through Explainable AI
• A store-and-forward cloud-based telemonitoring system for automatic assessing dysarthria evolution in neurological diseases from video-recording analysis
• Automatic 3D Registration of Dental CBCT and Face Scan Data using 2D Projection Images
• A Detailed Look At CNN-based Approaches In Facial Landmark Detection
• A Survey on Personalized Content Synthesis with Diffusion Models
• A Survey of Deep Facial Attribute Analysis
• Infinite 3D Landmarks: Improving Continuous 2D Facial Landmark Detection
• Quantum-Assisted Support Vector Regression
• A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset
• Benchmarking Shadow Removal for Facial Landmark Detection and Beyond
• Robotic Eye-in-hand Visual Servo Axially Aligning Nasopharyngeal Swabs with the Nasal Cavity
• Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking
• Real-time Ultrasound-enhanced Multimodal Imaging of Tongue using 3D Printable Stabilizer System: A Deep Learning Approach
• Unsupervised Performance Analysis of 3D Face Alignment with a Statistically Robust Confidence Test
• Multiple Face Analyses through Adversarial Learning
• Facial Landmark Correlation Analysis
• HOT-POT: Optimal Transport for Sparse Stereo Matching
• Domain Adaptation for Facial Expression Classifier via Domain Discrimination and Gradient Reversal
• Simultaneous regression and feature learning for facial landmarking
• Scalable Semi-supervised Landmark Localization for X-ray Images using Few-shot Deep Adaptive Graph