Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
arXiv:2105.08127
Abstract
Recent research has shown that numerous human-interpretable directions exist in the latent space of GANs. In this paper, we develop an automatic procedure for finding directions that lead to foreground-background image separation, and we use these directions to train an image segmentation model without human supervision. Our method is generator-agnostic, producing strong segmentation results with a wide range of different GAN architectures. Furthermore, by leveraging GANs pretrained on large datasets such as ImageNet, we are able to segment images from a range of domains without further training or finetuning. Evaluating our method on image segmentation benchmarks, we compare favorably to prior work while using neither human supervision nor access to the training data. Broadly, our results demonstrate that automatically extracting foreground-background structure from pretrained deep generative models can serve as a remarkably effective substitute for human supervision.
arXiv commentProject page and GitHub link: https://lukemelas.github.io/unsupervised-image-segmentation & https://github.com/lukemelas/unsupervised-image-segmentation
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- A Simple Framework for Contrastive Learning of Visual Representations
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Self-Attention Generative Adversarial Networks
- Conditional Image Synthesis With Auxiliary Classifier GANs
- ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
- Deeply supervised salient object detection with short connections
- DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
- Disentangling factors of variation in deep representations using adversarial training
- W-Net: A Deep Model for Fully Unsupervised Image Segmentation
- Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
- Closed-Form Factorization of Latent Semantics in GANs
- Object Discovery with a Copy-Pasting GAN
- Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data
- Unsupervised Domain Adaptation in Semantic Segmentation: a Review