Clearing the Skies: A deep network architecture for single-image rain removal

Abstract

We introduce a deep network architecture called DerainNet for removing rain streaks from an image. Based on the deep convolutional neural network (CNN), we directly learn the mapping relationship between rainy and clean image detail layers from data. Because we do not possess the ground truth corresponding to real-world rainy images, we synthesize images with rain for training. In contrast to other common strategies that increase depth or breadth of the network, we use image processing domain knowledge to modify the objective function and improve deraining with a modestly-sized CNN. Specifically, we train our DerainNet on the detail (high-pass) layer rather than in the image domain. Though DerainNet is trained on synthetic data, we find that the learned network translates very effectively to real-world images for testing. Moreover, we augment the CNN framework with image enhancement to improve the visual results. Compared with state-of-the-art single image de-raining methods, our method has improved rain removal and much faster computation time after network training.

References in corpus (1)

• Deep Learning in Neural Networks: An Overview

Cited by in corpus (86)

• DesnowNet: Context-Aware Deep Network for Snow Removal
• SAR Image Despeckling Using a Convolutional Neural Network
• Perceptual Adversarial Networks for Image-to-Image Transformation
• Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors
• Hyperspectral Image Classification with Markov Random Fields and a Convolutional Neural Network
• Image De-raining Using a Conditional Generative Adversarial Network
• Restormer: Efficient Transformer for High-Resolution Image Restoration
• FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors
• Rain Removal in Traffic Surveillance: Does it Matter?
• Pre-Trained Image Processing Transformer
• Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal
• Conditional Variational Image Deraining
• Density-aware Single Image De-raining using a Multi-stream Dense Network
• Rain Streak Removal for Single Image via Kernel Guided CNN
• Learning Converged Propagations with Deep Prior Ensemble for Image Enhancement
• Optical Fringe Patterns Filtering Based on Multi-Stage Convolution Neural Network
• A Survey on Rain Removal from Video and Single Image
• Confidence Measure Guided Single Image De-raining
• A Cascaded Convolutional Neural Network for X-ray Low-dose CT Image Denoising
• Multi-Stage Progressive Image Restoration
• Multi-Scale Progressive Fusion Network for Single Image Deraining
• Attentive Generative Adversarial Network for Raindrop Removal from a Single Image
• Deep-Masking Generative Network: A Unified Framework for Background Restoration from Superimposed Images
• From heavy rain removal to detail restoration: A faster and better network
• Deep Idempotent Network for Efficient Single Image Blind Deblurring
• Joint Transmission Map Estimation and Dehazing using Deep Networks
• Multi-scale Attentive Image De-raining Networks via Neural Architecture Search
• Exploring Overcomplete Representations for Single Image Deraining using CNNs
• Residual-Guide Feature Fusion Network for Single Image Deraining
• Gated Context Aggregation Network for Image Dehazing and Deraining
• UG${}^{2+}$ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments
• Physics-Based Generative Adversarial Models for Image Restoration and Beyond
• Investigating Task-driven Latent Feasibility for Nonconvex Image Modeling
• Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining
• Single Image Deraining using Scale-Aware Multi-Stage Recurrent Network
• A Model-driven Deep Neural Network for Single Image Rain Removal
• HINet: Half Instance Normalization Network for Image Restoration
• Learning Dual Convolutional Neural Networks for Low-Level Vision
• Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration
• DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking
• Single Image Deraining: From Model-Based to Data-Driven and Beyond
• DTDN: Dual-task De-raining Network
• EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining
• Progressive Image Deraining Networks: A Better and Simpler Baseline
• A Coarse-to-Fine Multi-stream Hybrid Deraining Network for Single Image Deraining
• Lightweight Pyramid Networks for Image Deraining
• SDNet: mutil-branch for single image deraining using swin
• RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining
• Robust Representation Learning with Feedback for Single Image Deraining
• Unfolding Taylor's Approximations for Image Restoration
• Unsupervised Single Image Deraining with Self-supervised Constraints
• Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation
• A^2Net: Adjacent Aggregation Networks for Image Raindrop Removal
• Structure-Preserving Deraining with Residue Channel Prior Guidance
• Memory-guided Image De-raining Using Time-Lapse Data
• Rethinking Image Deraining via Rain Streaks and Vapors
• An Effective Two-Branch Model-Based Deep Network for Single Image Deraining
• Semi-Supervised Image Deraining using Gaussian Processes
• Video Rain/Snow Removal by Transformed Online Multiscale Convolutional Sparse Coding
• Syn2Real Transfer Learning for Image Deraining using Gaussian Processes
• Physical Model Guided Deep Image Deraining
• Gradient Information Guided Deraining with A Novel Network and Adversarial Training
• A Deep Tree-Structured Fusion Model for Single Image Deraining
• Semi-Supervised Video Deraining with Dynamical Rain Generator
• Structural Residual Learning for Single Image Rain Removal
• Joint multi-dimensional dynamic attention and transformer for general image restoration
• Single Image Deraining with Continuous Rain Density Estimation
• Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution
• Non-local Channel Aggregation Network for Single Image Rain Removal
• ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media
• Subjective and Objective De-raining Quality Assessment Towards Authentic Rain Image
• Removing rain streaks by a linear model
• Rain Removal By Image Quasi-Sparsity Priors
• Rain Removal and Illumination Enhancement Done in One Go
• Directional Regularized Tensor Modeling for Video Rain Streaks Removal
• Residual Squeeze-and-Excitation Network for Fast Image Deraining
• Towards Safer Transportation: a self-supervised learning approach for traffic video deraining
• From Rain Generation to Rain Removal
• Morphological Networks for Image De-raining
• Towards a Unified Approach to Single Image Deraining and Dehazing
• Investigating Customization Strategies and Convergence Behaviors of Task-specific ADMM
• Deep Single Image Deraining Via Estimating Transmission and Atmospheric Light in rainy Scenes
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
• Deep Image Deraining Via Intrinsic Rainy Image Priors and Multi-scale Auxiliary Decoding
• Image Deraining and Denoising Convolutional Neural Network ForAutonomous Driving
• Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining