Semantic Photo Manipulation with a Generative Image Prior

Abstract

Despite the recent success of GANs in synthesizing images conditioned on inputs such as a user sketch, text, or semantic labels, manipulating the high-level attributes of an existing natural photograph with GANs is challenging for two reasons. First, it is hard for GANs to precisely reproduce an input image. Second, after manipulation, the newly synthesized pixels often do not fit the original image. In this paper, we address these issues by adapting the image prior learned by GANs to image statistics of an individual image. Our method can accurately reconstruct the input image and synthesize new content, consistent with the appearance of the input image. We demonstrate our interactive system on several semantic image editing tasks, including synthesizing new objects consistent with background, removing unwanted objects, and changing the appearance of an object. Quantitative and qualitative comparisons against several existing methods demonstrate the effectiveness of our method.

SIGGRAPH 2019

References in corpus (1)

• Deep Bilateral Learning for Real-Time Image Enhancement

Cited by in corpus (18)

• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• Adversarial Text-to-Image Synthesis: A Review
• Break-A-Scene: Extracting Multiple Concepts from a Single Image
• Constrained Graphic Layout Generation via Latent Optimization
• SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
• E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
• StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation
• COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles
• Guided Fine-Tuning for Large-Scale Material Transfer
• GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information
• ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis
• Guided Co-Modulated GAN for 360° Field of View Extrapolation
• Stable Flow: Vital Layers for Training-Free Image Editing
• Example Forgetting: A Novel Approach to Explain and Interpret Deep Neural Networks in Seismic Interpretation
• Interactive Neural Painting
• TilinGNN: Learning to Tile with Self-Supervised Graph Neural Network
• Towards Controllable and Photorealistic Region-wise Image Manipulation
• SwipeGANSpace: Swipe-to-Compare Image Generation via Efficient Latent Space Exploration