On Equivariant and Invariant Learning of Object Landmark Representations
arXiv:2006.14787
Abstract
Given a collection of images, humans are able to discover landmarks by modeling the shared geometric structure across instances. This idea of geometric equivariance has been widely used for the unsupervised discovery of object landmark representations. In this paper, we develop a simple and effective approach by combining instance-discriminative and spatially-discriminative contrastive learning. We show that when a deep network is trained to be invariant to geometric and photometric transformations, representations emerge from its intermediate layers that are highly predictive of object landmarks. Stacking these across layers in a "hypercolumn" and projecting them using spatially-contrastive learning further improves their performance on matching and few-shot landmark regression tasks. We also present a unified view of existing equivariant and invariant representation learning approaches through the lens of contrastive learning, shedding light on the nature of invariances learned. Experiments on standard benchmarks for landmark learning, as well as a new challenging one we propose, show that the proposed approach surpasses prior state-of-the-art.
arXiv commentProject Page: https://people.cs.umass.edu/~zezhoucheng/contrastive_landmark Code: https://github.com/cvl-umass/ContrastLandmark
References in corpus (16)
- Representation Learning with Contrastive Predictive Coding
- The information bottleneck method
- Improved Baselines with Momentum Contrastive Learning
- Learning deep representations by mutual information estimation and maximization
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Group Equivariant Convolutional Networks
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
- Learning Deep Representation for Face Alignment with Auxiliary Attributes
- Self-supervised learning of a facial attribute embedding from video
- A critical analysis of self-supervision, or what we can learn from a single image
- Unsupervised Learning of Object Landmarks through Conditional Image Generation
- Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
- Unsupervised Learning of Dense Visual Representations
- Unsupervised Landmark Learning from Unpaired Data
- Object landmark discovery through unsupervised adaptation
- BRULÈ: Barycenter-Regularized Unsupervised Landmark Extraction