Learning Deep Representation for Face Alignment with Auxiliary Attributes

Abstract

In this study, we show that landmark detection or face alignment task is not a single and independent problem. Instead, its robustness can be greatly improved with auxiliary information. Specifically, we jointly optimize landmark detection together with the recognition of heterogeneous but subtly correlated facial attributes, such as gender, expression, and appearance attributes. This is non-trivial since different attribute inference tasks have different learning difficulties and convergence rates. To address this problem, we formulate a novel tasks-constrained deep model, which not only learns the inter-task correlation but also employs dynamic task coefficients to facilitate the optimization convergence when learning multiple complex tasks. Extensive evaluations show that the proposed task-constrained learning (i) outperforms existing face alignment methods, especially in dealing with faces with severe occlusion and pose variation, and (ii) reduces model complexity drastically compared to the state-of-the-art methods based on cascaded deep model.

to be published in the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

References in corpus (5)

• A Convex Formulation for Learning Task Relationships in Multi-Task Learning
• Recover Canonical-View Faces in the Wild with Deep Neural Networks
• Surpassing Human-Level Face Verification Performance on LFW with GaussianFace
• Deep Learning Multi-View Representation for Face Recognition
• Deep Regression for Face Alignment

Cited by in corpus (76)

• Facial Landmark Detection: a Literature Survey
• Face Alignment Across Large Poses: A 3D Solution
• Face Alignment in Full Pose Range: A 3D Total Solution
• Text-Attentional Convolutional Neural Networks for Scene Text Detection
• Image Aesthetic Assessment: An Experimental Survey
• Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition
• Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
• J$\hat{\text{A}}$A-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention
• Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild
• Drivers Drowsiness Detection using Condition-Adaptive Representation Learning Framework
• Self-supervised learning of a facial attribute embedding from video
• Locally-Supervised Deep Hybrid Model for Scene Recognition
• Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
• Multi-task head pose estimation in-the-wild
• Structured Landmark Detection via Topology-Adapting Deep Graph Learning
• Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection
• WIDER FACE: A Face Detection Benchmark
• Occlusion Coherence: Detecting and Localizing Occluded Faces
• Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
• Look, Listen and Learn - A Multimodal LSTM for Speaker Identification
• ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild
• Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
• Deep Appearance Models: A Deep Boltzmann Machine Approach for Face Modeling
• Respiratory Rate Estimation from Face Videos
• Look at Boundary: A Boundary-Aware Face Alignment Algorithm
• Learning Social Relation Traits from Face Images
• Facial Landmark Detection with Tweaked Convolutional Neural Networks
• Dynamic Multi-Task Learning for Face Recognition with Facial Expression
• Transferring Landmark Annotations for Cross-Dataset Face Alignment
• MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution
• LOTR: Face Landmark Localization Using Localization Transformer
• Deep Multi-Center Learning for Face Alignment
• Facial Landmarks Localization using Cascaded Neural Networks
• Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry
• From Facial Expression Recognition to Interpersonal Relation Prediction
• Single-Network Whole-Body Pose Estimation
• Learning deep representation from coarse to fine for face alignment
• Unconstrained Facial Action Unit Detection via Latent Feature Domain
• Training with the Invisibles: Obfuscating Images to Share Safely for Learning Visual Recognition Models
• Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification
• Unsupervised Landmark Learning from Unpaired Data
• AnchorFace: An Anchor-based Facial Landmark Detector Across Large Poses
• Learning to Impute: A General Framework for Semi-supervised Learning
• A Detailed Look At CNN-based Approaches In Facial Landmark Detection
• PropagationNet: Propagate Points to Curve to Learn Structure Information
• Unsupervised Tracklet Person Re-Identification
• Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies
• Whole-Body Human Pose Estimation in the Wild
• Towards Highly Accurate and Stable Face Alignment for High-Resolution Videos
• Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detection
• Predicting Personal Traits from Facial Images using Convolutional Neural Networks Augmented with Facial Landmark Information
• A Deep Multi-task Learning Approach to Skin Lesion Classification
• Learning Symmetry Consistent Deep CNNs for Face Completion
• Sub-pixel face landmarks using heatmaps and a bag of tricks
• From Generalized zero-shot learning to long-tail with class descriptors
• Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking
• LDDMM-Face: Large Deformation Diffeomorphic Metric Learning for Flexible and Consistent Face Alignment
• Towards Omni-Supervised Face Alignment for Large Scale Unlabeled Videos
• Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks
• Impact of facial landmark localization on facial expression recognition
• Multistage Model for Robust Face Alignment Using Deep Neural Networks
• Algorithmic Fairness Datasets: the Story so Far
• MobileFAN: Transferring Deep Hidden Representation for Face Alignment
• Deep Architectures for Face Attributes
• Automatic Detection and Classification of Waste Consumer Medications for Proper Management and Disposal
• L2GSCI: Local to Global Seam Cutting and Integrating for Accurate Face Contour Extraction
• Unsupervised Part Discovery via Feature Alignment
• Facial Landmark Correlation Analysis
• On Equivariant and Invariant Learning of Object Landmark Representations
• Scalable Semi-supervised Landmark Localization for X-ray Images using Few-shot Deep Adaptive Graph
• CoopSubNet: Cooperating Subnetwork for Data-Driven Regularization of Deep Networks under Limited Training Budgets
• Fast Landmark Localization with 3D Component Reconstruction and CNN for Cross-Pose Recognition
• Unsupervised Part Segmentation through Disentangling Appearance and Shape
• Cross-Task Representation Learning for Anatomical Landmark Detection
• Cross-modal Multi-task Learning for Graphic Recognition of Caricature Face
• The Blessing and the Curse of the Noise behind Facial Landmark Annotations