Structural Consistency and Controllability for Diverse Colorization
arXiv:1809.02129
Abstract
Colorizing a given gray-level image is an important task in the media and advertising industry. Due to the ambiguity inherent to colorization (many shades are often plausible), recent approaches started to explicitly model diversity. However, one of the most obvious artifacts, structural inconsistency, is rarely considered by existing methods which predict chrominance independently for every pixel. To address this issue, we develop a conditional random field based variational auto-encoder formulation which is able to achieve diversity while taking into account structural consistency. Moreover, we introduce a controllability mecha- nism that can incorporate external constraints from diverse sources in- cluding a user interface. Compared to existing baselines, we demonstrate that our method obtains more diverse and globally consistent coloriza- tions on the LFW, LSUN-Church and ILSVRC-2015 datasets.
arXiv commentAccepted to ECCV 2018
References in corpus (8)
- Auto-Encoding Variational Bayes
- Conditional Image Generation with PixelCNN Decoders
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- Toward Multimodal Image-to-Image Translation
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Creativity: Generating Diverse Questions using Variational Autoencoders
- Unsupervised Diverse Colorization via Generative Adversarial Networks