SEIGAN: Towards Compositional Image Generation by Simultaneously Learning to Segment, Enhance, and Inpaint
arXiv:1811.07630
Abstract
We present a novel approach to image manipulation and understanding by simultaneously learning to segment object masks, paste objects to another background image, and remove them from original images. For this purpose, we develop a novel generative model for compositional image generation, SEIGAN (Segment-Enhance-Inpaint Generative Adversarial Network), which learns these three operations together in an adversarial architecture with additional cycle consistency losses. To train, SEIGAN needs only bounding box supervision and does not require pairing or ground truth masks. SEIGAN produces better generated images (evaluated by human assessors) than other approaches and produces high-quality segmentation masks, improving over other adversarially trained approaches and getting closer to the results of fully supervised training.
References in corpus (7)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
- W-Net: A Deep Model for Fully Unsupervised Image Segmentation
- CycleGAN, a Master of Steganography
- Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
- TextureGAN: Controlling Deep Image Synthesis with Texture Patches
Cited by in corpus (9)
- Unsupervised Object Segmentation by Redrawing
- Behind the leaves -- Estimation of occluded grapevine berries with conditional generative adversarial networks
- Emergence of Object Segmentation in Perturbed Generative Models
- Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps
- BRULÈ: Barycenter-Regularized Unsupervised Landmark Extraction
- TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors
- GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettes
- Instance Separation Emerges from Inpainting
- Visual Boundary Knowledge Translation for Foreground Segmentation