Spatially Controllable Image Synthesis with Internal Representation Collaging

Abstract

We present a novel CNN-based image editing strategy that allows the user to change the semantic information of an image over an arbitrary region by manipulating the feature-space representation of the image in a trained GAN model. We will present two variants of our strategy: (1) spatial conditional batch normalization (sCBN), a type of conditional batch normalization with user-specifiable spatial weight maps, and (2) feature-blending, a method of directly modifying the intermediate features. Our methods can be used to edit both artificial image and real image, and they both can be used together with any GAN with conditional normalization layers. We will demonstrate the power of our method through experiments on various types of GANs trained on different datasets. Code will be available at https://github.com/pfnet-research/neural-collage.

References in corpus (8)

• Conditional Generative Adversarial Nets
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Toward Multimodal Image-to-Image Translation
• A Learned Representation For Artistic Style
• Modulating early visual processing by language
• Multi-style Generative Network for Real-time Transfer

Cited by in corpus (11)

• Alias-Free Generative Adversarial Networks
• Using latent space regression to analyze and leverage compositionality in GANs
• Editing in Style: Uncovering the Local Semantics of GANs
• Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks
• Controllable Image Synthesis via SegVAE
• Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN
• Low-Rank Subspaces in GANs
• Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
• StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGAN
• MixSyn: Learning Composition and Style for Multi-Source Image Synthesis
• Linear Semantics in Generative Adversarial Networks