GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation
arXiv:2112.01036
Abstract
Segmenting an image into its parts is a frequent preprocess for high-level vision tasks such as image editing. However, annotating masks for supervised training is expensive. Weakly-supervised and unsupervised methods exist, but they depend on the comparison of pairs of images, such as from multi-views, frames of videos, and image augmentation, which limits their applicability. To address this, we propose a GAN-based approach that generates images conditioned on latent masks, thereby alleviating full or weak annotations required in previous approaches. We show that such mask-conditioned image generation can be learned faithfully when conditioning the masks in a hierarchical manner on latent keypoints that define the position of parts explicitly. Without requiring supervision of masks or points, this strategy increases robustness to viewpoint and object positions changes. It also lets us generate image-mask pairs for training a segmentation network, which outperforms the state-of-the-art unsupervised segmentation methods on established benchmarks.
arXiv commentCVPR 2022
References in corpus (13)
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- Which Training Methods for GANs do actually Converge?
- Conditional Positional Encodings for Vision Transformers
- Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
- Unsupervised Learning of Object Keypoints for Perception and Control
- Unsupervised Learning of Object Landmarks through Conditional Image Generation
- How Much Position Information Do Convolutional Neural Networks Encode?
- Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction
- How Can CNNs Use Image Position for Segmentation?
- Unsupervised Landmark Learning from Unpaired Data
- Object Segmentation Without Labels with Large-Scale Generative Models