A Recurrent Encoder-Decoder Network for Sequential Face Alignment

Abstract

We propose a novel recurrent encoder-decoder network model for real-time video-based face alignment. Our proposed model predicts 2D facial point maps regularized by a regression loss, while uniquely exploiting recurrent learning at both spatial and temporal dimensions. At the spatial level, we add a feedback loop connection between the combined output response map and the input, in order to enable iterative coarse-to-fine face alignment using a single network model. At the temporal level, we first decouple the features in the bottleneck of the network into temporal-variant factors, such as pose and expression, and temporal-invariant factors, such as identity information. Temporal recurrent learning is then applied to the decoupled temporal-variant features, yielding better generalization and significantly more accurate results at test time. We perform a comprehensive experimental analysis, showing the importance of each component of our proposed model, as well as superior results over the state-of-the-art in standard datasets.

European Conference on Computer Vision (ECCV), 2016

References in corpus (4)

• Conditional Random Fields as Recurrent Neural Networks
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Describing Videos by Exploiting Temporal Structure
• Do Convnets Learn Correspondence?

Cited by in corpus (16)

• Bilateral Multi-Perspective Matching for Natural Language Sentences
• Density-aware Single Image De-raining using a Multi-stream Dense Network
• Style Aggregated Network for Facial Landmark Detection
• Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
• CR-GAN: Learning Complete Representations for Multi-view Generation
• Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors
• Pose-Invariant Face Alignment with a Single CNN
• Deep Learning For Face Recognition: A Critical Analysis
• Learning Dense Facial Correspondences in Unconstrained Images
• Word Mover's Embedding: From Word2Vec to Document Embedding
• InverseFaceNet: Deep Monocular Inverse Face Rendering
• Deep Texture Manifold for Ground Terrain Recognition
• Interspecies Knowledge Transfer for Facial Keypoint Detection
• Towards Highly Accurate and Stable Face Alignment for High-Resolution Videos
• Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution
• Neural Stochastic Block Model & Scalable Community-Based Graph Learning