3D-Aided Data Augmentation for Robust Face Understanding

Abstract

Data augmentation has been highly effective in narrowing the data gap and reducing the cost for human annotation, especially for tasks where ground truth labels are difficult and expensive to acquire. In face recognition, large pose and illumination variation of face images has been a key factor for performance degradation. However, human annotation for the various face understanding tasks including face landmark localization, face attributes classification and face recognition under these challenging scenarios are highly costly to acquire. Therefore, it would be desirable to perform data augmentation for these cases. But simple 2D data augmentation techniques on the image domain are not able to satisfy the requirement of these challenging cases. As such, 3D face modeling, in particular, single image 3D face modeling, stands a feasible solution for these challenging conditions beyond 2D based data augmentation. To this end, we propose a method that produces realistic 3D augmented images from multiple viewpoints with different illumination conditions through 3D face modeling, each associated with geometrically accurate face landmarks, attributes and identity information. Experiments demonstrate that the proposed 3D data augmentation method significantly improves the performance and robustness of various face understanding tasks while achieving state-of-arts on multiple benchmarks.

References in corpus (7)

• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Data Augmentation Generative Adversarial Networks
• Multicolumn Networks for Face Recognition
• PFLD: A Practical Facial Landmark Detector
• Attributes for Improved Attributes: A Multi-Task Network for Attribute Classification
• Load Balanced GANs for Multi-view Face Image Synthesis
• Dataset Augmentation for Pose and Lighting Invariant Face Recognition