Image De-raining Using a Conditional Generative Adversarial Network

Abstract

Severe weather conditions such as rain and snow adversely affect the visual quality of images captured under such conditions thus rendering them useless for further usage and sharing. In addition, such degraded images drastically affect performance of vision systems. Hence, it is important to solve the problem of single image de-raining/de-snowing. However, this is a difficult problem to solve due to its inherent ill-posed nature. Existing approaches attempt to introduce prior information to convert it into a well-posed problem. In this paper, we investigate a new point of view in addressing the single image de-raining problem. Instead of focusing only on deciding what is a good prior or a good framework to achieve good quantitative and qualitative performance, we also ensure that the de-rained image itself does not degrade the performance of a given computer vision algorithm such as detection and classification. In other words, the de-rained result should be indistinguishable from its corresponding clear image to a given discriminator. This criterion can be directly incorporated into the optimization framework by using the recently introduced conditional generative adversarial networks (GANs). To minimize artifacts introduced by GANs and ensure better visual quality, a new refined loss function is introduced. Based on this, we propose a novel single image de-raining method called Image De-raining Conditional General Adversarial Network (ID-CGAN), which considers quantitative, visual and also discriminative performance into the objective function. Experiments evaluated on synthetic images and real images show that the proposed method outperforms many recent state-of-the-art single image de-raining methods in terms of quantitative and visual performance.

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• Energy-based Generative Adversarial Network
• Perceptual Adversarial Networks for Image-to-Image Transformation
• Generative Image Inpainting with Contextual Attention
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• Rain Streak Removal for Single Image via Kernel Guided CNN
• Residual-Guide Feature Fusion Network for Single Image Deraining
• Task Specific Adversarial Cost Function
• Lightweight Pyramid Networks for Image Deraining

Cited by in corpus (27)

• A Survey of Recent Advances in CNN-based Single Image Crowd Counting and Density Estimation
• Perceptual Adversarial Networks for Image-to-Image Transformation
• Sharpness-aware Low dose CT denoising using conditional generative adversarial network
• Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification
• Filmy Cloud Removal on Satellite Imagery with Multispectral Conditional Generative Adversarial Nets
• Rain Removal in Traffic Surveillance: Does it Matter?
• Tell Me Where to Look: Guided Attention Inference Network
• Confidence Measure Guided Single Image De-raining
• Residual-Guide Feature Fusion Network for Single Image Deraining
• Physics-Based Generative Adversarial Models for Image Restoration and Beyond
• Learning Dual Convolutional Neural Networks for Low-Level Vision
• DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking
• Single Image Deraining: From Model-Based to Data-Driven and Beyond
• Semi-DerainGAN: A New Semi-supervised Single Image Deraining Network
• Single Image Haze Removal using a Generative Adversarial Network
• Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset
• Fast Single Image Rain Removal via a Deep Decomposition-Composition Network
• A Coarse-to-Fine Multi-stream Hybrid Deraining Network for Single Image Deraining
• Deep learning-based conditional inpainting for restoration of artifact-affected 4D CT images
• High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks
• High Resolution Millimeter Wave Imaging For Self-Driving Cars
• MBA-RainGAN: Multi-branch Attention Generative Adversarial Network for Mixture of Rain Removal from Single Images
• Learning Compositional Visual Concepts with Mutual Consistency
• Video Rain/Snow Removal by Transformed Online Multiscale Convolutional Sparse Coding
• Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution
• EnsNet: Ensconce Text in the Wild
• Deep Image Deraining Via Intrinsic Rainy Image Priors and Multi-scale Auxiliary Decoding