LUCSS: Language-based User-customized Colourization of Scene Sketches
arXiv:1808.10544
Abstract
We introduce LUCSS, a language-based system for interactive col- orization of scene sketches, based on their semantic understanding. LUCSS is built upon deep neural networks trained via a large-scale repository of scene sketches and cartoon-style color images with text descriptions. It con- sists of three sequential modules. First, given a scene sketch, the segmenta- tion module automatically partitions an input sketch into individual object instances. Next, the captioning module generates the text description with spatial relationships based on the instance-level segmentation results. Fi- nally, the interactive colorization module allows users to edit the caption and produce colored images based on the altered caption. Our experiments show the effectiveness of our approach and the desirability of its compo- nents to alternative choices.
References in corpus (22)
- Adam: A Method for Stochastic Optimization
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Generative Adversarial Text to Image Synthesis
- Pixel Recurrent Neural Networks
- Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- A Neural Representation of Sketch Drawings
- Holistically-Nested Edge Detection
- Simultaneous Detection and Segmentation
- Learning a Recurrent Visual Representation for Image Caption Generation
- Amortised MAP Inference for Image Super-resolution
- Sketch-based 3D Shape Retrieval using Convolutional Neural Networks
- Proposal-free Network for Instance-level Object Segmentation
- Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks
- Convolutional Sketch Inversion
- Outline Colorization through Tandem Adversarial Networks
- Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
- SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis
- TextureGAN: Controlling Deep Image Synthesis with Texture Patches
- Watch What You Just Said: Image Captioning with Text-Conditional Attention
- Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
- SketchyScene: Richly-Annotated Scene Sketches