Unsupervised Part Discovery from Contrastive Reconstruction
arXiv:2111.06349
Abstract
The goal of self-supervised visual representation learning is to learn strong, transferable image representations, with the majority of research focusing on object or scene level. On the other hand, representation learning at part level has received significantly less attention. In this paper, we propose an unsupervised approach to object part discovery and segmentation and make three contributions. First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts. Secondly, prior work argues for reconstructing or clustering pre-computed features as a proxy to parts; we show empirically that this alone is unlikely to find meaningful parts; mainly because of their low resolution and the tendency of classification networks to spatially smear out information. We suggest that image reconstruction at the level of pixels can alleviate this problem, acting as a complementary cue. Lastly, we show that the standard evaluation based on keypoint regression does not correlate well with segmentation quality and thus introduce different metrics, NMI and ARI, that better characterize the decomposition of objects into parts. Our method yields semantic parts which are consistent across fine-grained but visually distinct categories, outperforming the state of the art on three benchmark datasets. Code is available at the project page: https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/.
arXiv commentNeurIPS 2021. Project page: https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/
References in corpus (19)
- Auto-Encoding Variational Bayes
- Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
- Learning deep representations by mutual information estimation and maximization
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Multi-Object Representation Learning with Iterative Variational Inference
- MONet: Unsupervised Scene Decomposition and Representation
- GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
- Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
- SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
- Learning Physical Graph Representations from Visual Scenes
- Deep Learning for Semantic Part Segmentation with High-Level Guidance
- Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
- SCALOR: Generative World Models with Scalable Object Representations
- Unsupervised Discovery of Parts, Structure, and Dynamics
- Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
- DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort
- Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation
- SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition
- Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery