TextureGAN: Controlling Deep Image Synthesis with Texture Patches
arXiv:1706.02823
Abstract
In this paper, we investigate deep image synthesis guided by sketch, color, and texture. Previous image synthesis methods can be controlled by sketch and color strokes but we are the first to examine texture control. We allow a user to place a texture patch on a sketch at arbitrary locations and scales to control the desired output texture. Our generative network learns to synthesize objects consistent with these texture suggestions. To achieve this, we develop a local texture loss in addition to adversarial and content loss to train the generative network. We conduct experiments using sketches generated from real images and textures sampled from a separate texture database and results show that our proposed algorithm is able to generate plausible images that are faithful to user controls. Ablation studies show that our proposed pipeline can generate more realistic images than adapting existing methods directly.
arXiv commentCVPR 2018 spotlight
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Generative Adversarial Text to Image Synthesis
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
- Texture Synthesis Using Convolutional Neural Networks
- Deep Human Parsing with Active Template Regression
- Dilated Residual Networks
- Least Squares Generative Adversarial Networks
- Multi-style Generative Network for Real-time Transfer
- Texture Synthesis with Spatial Generative Adversarial Networks
- Learning Representations for Automatic Colorization
- Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks
- Convolutional Sketch Inversion
- Diversified Texture Synthesis with Feed-forward Networks
- Learning Texture Manifolds with the Periodic Spatial GAN
Cited by in corpus (7)
- Toward Multimodal Image-to-Image Translation
- SEIGAN: Towards Compositional Image Generation by Simultaneously Learning to Segment, Enhance, and Inpaint
- CFA-Net: Controllable Face Anonymization Network with Identity Representation Manipulation
- LUCSS: Language-based User-customized Colourization of Scene Sketches
- BachGAN: High-Resolution Image Synthesis from Salient Object Layout
- Smart, Sparse Contours to Represent and Edit Images
- Specular-to-Diffuse Translation for Multi-View Reconstruction