BRULÈ: Barycenter-Regularized Unsupervised Landmark Extraction

Abstract

Unsupervised retrieval of image features is vital for many computer vision tasks where the annotation is missing or scarce. In this work, we propose a new unsupervised approach to detect the landmarks in images, validating it on the popular task of human face key-points extraction. The method is based on the idea of auto-encoding the wanted landmarks in the latent space while discarding the non-essential information (and effectively preserving the interpretability). The interpretable latent space representation (the bottleneck containing nothing but the wanted key-points) is achieved by a new two-step regularization approach. The first regularization step evaluates transport distance from a given set of landmarks to some average value (the barycenter by Wasserstein distance). The second regularization step controls deviations from the barycenter by applying random geometric deformations synchronously to the initial image and to the encoded landmarks. We demonstrate the effectiveness of the approach both in unsupervised and semi-supervised training scenarios using 300-W, CelebA, and MAFL datasets. The proposed regularization paradigm is shown to prevent overfitting, and the detection quality is shown to improve beyond the state-of-the-art face models.

arXiv comment10 main pages with 6 figures and 1 Table, 14 pages total with 6 supplementary figures. I.B. and N.B. contributed equally. D.V.D. is corresponding author

References in corpus (8)

• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Stacked Hourglass Networks for Human Pose Estimation
• Pyramid Scene Parsing Network
• W-Net: A Deep Model for Fully Unsupervised Image Segmentation
• Self-supervised learning of a facial attribute embedding from video
• Unsupervised Learning of Object Landmarks through Conditional Image Generation
• SEIGAN: Towards Compositional Image Generation by Simultaneously Learning to Segment, Enhance, and Inpaint

Cited by in corpus (2)

• Optimal MRI Undersampling Patterns for Ultimate Benefit of Medical Vision Tasks
• On Equivariant and Invariant Learning of Object Landmark Representations