Unsupervised Single Image Deraining with Self-supervised Constraints

Abstract

Most existing single image deraining methods require learning supervised models from a large set of paired synthetic training data, which limits their generality, scalability and practicality in real-world multimedia applications. Besides, due to lack of labeled-supervised constraints, directly applying existing unsupervised frameworks to the image deraining task will suffer from low-quality recovery. Therefore, we propose an Unsupervised Deraining Generative Adversarial Network (UD-GAN) to tackle above problems by introducing self-supervised constraints from the intrinsic statistics of unpaired rainy and clean images. Specifically, we firstly design two collaboratively optimized modules, namely Rain Guidance Module (RGM) and Background Guidance Module (BGM), to take full advantage of rainy image characteristics: The RGM is designed to discriminate real rainy images from fake rainy images which are created based on outputs of the generator with BGM. Simultaneously, the BGM exploits a hierarchical Gaussian-Blur gradient error to ensure background consistency between rainy input and de-rained output. Secondly, a novel luminance-adjusting adversarial loss is integrated into the clean image discriminator considering the built-in luminance difference between real clean images and derained images. Comprehensive experiment results on various benchmarking datasets and different training settings show that UD-GAN outperforms existing image deraining methods in both quantitative and qualitative comparisons.

arXiv comment10 pages, 8 figures

References in corpus (14)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Deep Residual Learning for Image Recognition
• Multi-Scale Context Aggregation by Dilated Convolutions
• Improved Techniques for Training GANs
• Clearing the Skies: A deep network architecture for single-image rain removal
• DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
• Unsupervised Visual Representation Learning by Context Prediction
• Unsupervised Learning of Visual Representations using Videos
• Learning to See by Moving
• Rain Streak Removal for Single Image via Kernel Guided CNN
• Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
• Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks
• Unsupervised Part-based Weighting Aggregation of Deep Convolutional Features for Image Retrieval
• Self-supervised learning of visual features through embedding images into text topic spaces