SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

Abstract

Synthesizing realistic images from human drawn sketches is a challenging problem in computer graphics and vision. Existing approaches either need exact edge maps, or rely on retrieval of existing photographs. In this work, we propose a novel Generative Adversarial Network (GAN) approach that synthesizes plausible images from 50 categories including motorcycles, horses and couches. We demonstrate a data augmentation technique for sketches which is fully automatic, and we show that the augmented data is helpful to our task. We introduce a new network building block suitable for both the generator and discriminator which improves the information flow by injecting the input image at multiple scales. Compared to state-of-the-art image translation methods, our approach generates more realistic images and achieves significantly higher Inception Scores.

Accepted to CVPR 2018

References in corpus (6)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Improved Training of Wasserstein GANs
• Focal Loss for Dense Object Detection
• A Learned Representation For Artistic Style
• Sketch-based 3D Shape Retrieval using Convolutional Neural Networks

Cited by in corpus (7)

• Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Human-Like Sketches
• LUCSS: Language-based User-customized Colourization of Scene Sketches
• Jamdani Motif Generation using Conditional GAN
• Attribute-Driven Spontaneous Motion in Unpaired Image Translation
• Exploring Crowd Co-creation Scenarios for Sketches
• LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network
• Systematic Analysis of Image Generation using GANs