Impact of facial landmark localization on facial expression recognition

Abstract

Although facial landmark localization (FLL) approaches are becoming increasingly accurate for characterizing facial regions, one question remains unanswered: what is the impact of these approaches on subsequent related tasks? In this paper, the focus is put on facial expression recognition (FER), where facial landmarks are used for face registration, which is a common usage. Since the most used datasets for facial landmark localization do not allow for a proper measurement of performance according to the different difficulties (e.g., pose, expression, illumination, occlusion, motion blur), we also quantify the performance of recent approaches in the presence of head pose variations and facial expressions. Finally, a study of the impact of these approaches on FER is conducted. We show that the landmark accuracy achieved so far optimizing the conventional Euclidean distance does not necessarily guarantee a gain in performance for FER. To deal with this issue, we propose a new evaluation metric for FLL adapted to FER.

References in corpus (11)

• Bootstrap your own latent: A new approach to self-supervised Learning
• Deep Facial Expression Recognition: A Survey
• Learning Face Representation from Scratch
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• Face Alignment Across Large Poses: A 3D Solution
• Face Alignment in Full Pose Range: A 3D Total Solution
• A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild"
• Advanced local motion patterns for macro and micro facial expression recognition
• Supervision by Registration and Triangulation for Landmark Detection