SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network

Abstract

Given a grayscale photograph, the colorization system estimates a visually plausible colorful image. Conventional methods often use semantics to colorize grayscale images. However, in these methods, only classification semantic information is embedded, resulting in semantic confusion and color bleeding in the final colorized image. To address these issues, we propose a fully automatic Saliency Map-guided Colorization with Generative Adversarial Network (SCGAN) framework. It jointly predicts the colorization and saliency map to minimize semantic confusion and color bleeding in the colorized image. Since the global features from pre-trained VGG-16-Gray network are embedded to the colorization encoder, the proposed SCGAN can be trained with much less data than state-of-the-art methods to achieve perceptually reasonable colorization. In addition, we propose a novel saliency map-based guidance method. Branches of the colorization decoder are used to predict the saliency map as a proxy target. Moreover, two hierarchical discriminators are utilized for the generated colorization and saliency map, respectively, in order to strengthen visual perception performance. The proposed system is evaluated on ImageNet validation set. Experimental results show that SCGAN can generate more reasonable colorized images than state-of-the-art techniques.

accepted by IEEE Transactions on Circuits and Systems for Video Technology

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Spectral Normalization for Generative Adversarial Networks
• Deeply supervised salient object detection with short connections
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Review of Visual Saliency Detection with Comprehensive Information
• Fake Colorized Image Detection
• Salient Object Detection via Augmented Hypotheses
• Pixel-level Semantics Guided Image Colorization

Cited by in corpus (4)

• VCGAN: Video Colorization with Hybrid Generative Adversarial Network
• SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation
• BC-GAN: A Generative Adversarial Network for Synthesizing a Batch of Collocated Clothing
• Transforming Color: A Novel Image Colorization Method