Rain Removal in Traffic Surveillance: Does it Matter?

Abstract

Varying weather conditions, including rainfall and snowfall, are generally regarded as a challenge for computer vision algorithms. One proposed solution to the challenges induced by rain and snowfall is to artificially remove the rain from images or video using rain removal algorithms. It is the promise of these algorithms that the rain-removed image frames will improve the performance of subsequent segmentation and tracking algorithms. However, rain removal algorithms are typically evaluated on their ability to remove synthetic rain on a small subset of images. Currently, their behavior is unknown on real-world videos when integrated with a typical computer vision pipeline. In this paper, we review the existing rain removal algorithms and propose a new dataset that consists of 22 traffic surveillance sequences under a broad variety of weather conditions that all include either rain or snowfall. We propose a new evaluation protocol that evaluates the rain removal algorithms on their ability to improve the performance of subsequent segmentation, instance segmentation, and feature tracking algorithms under rain and snow. If successful, the de-rained frames of a rain removal algorithm should improve segmentation performance and increase the number of accurately tracked features. The results show that a recent single-frame-based rain removal algorithm increases the segmentation performance by 19.7% on our proposed dataset, but it eventually decreases the feature tracking performance and showed mixed results with recent instance segmentation methods. However, the best video-based rain removal algorithm improves the feature tracking accuracy by 7.72%.

arXiv commentPublished in IEEE Transactions on Intelligent Transportation Systems

References in corpus (6)

• The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices
• Multi-Scale Context Aggregation by Dilated Convolutions
• Clearing the Skies: A deep network architecture for single-image rain removal
• Image De-raining Using a Conditional Generative Adversarial Network
• Fully Convolutional Instance-aware Semantic Segmentation
• The AAU Multimodal Annotation Toolboxes: Annotating Objects in Images and Videos

Cited by in corpus (12)

• FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors
• Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
• Rain Streak Removal for Single Image via Kernel Guided CNN
• DAWN: Vehicle Detection in Adverse Weather Nature Dataset
• A Survey on Rain Removal from Video and Single Image
• Mixture Domain Adaptation to Improve Semantic Segmentation in Real-World Surveillance
• Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
• Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation
• Structural Residual Learning for Single Image Rain Removal
• Gradient Information Guided Deraining with A Novel Network and Adversarial Training
• Task-Driven Deep Image Enhancement Network for Autonomous Driving in Bad Weather
• Deep Single Image Deraining Via Estimating Transmission and Atmospheric Light in rainy Scenes