UnShadowNet: Illumination Critic Guided Contrastive Learning For Shadow Removal

Abstract

Shadows are frequently encountered natural phenomena that significantly hinder the performance of computer vision perception systems in practical settings, e.g., autonomous driving. A solution to this would be to eliminate shadow regions from the images before the processing of the perception system. Yet, training such a solution requires pairs of aligned shadowed and non-shadowed images which are difficult to obtain. We introduce a novel weakly supervised shadow removal framework UnShadowNet trained using contrastive learning. It is composed of a DeShadower network responsible for the removal of the extracted shadow under the guidance of an Illumination network which is trained adversarially by the illumination critic and a Refinement network to further remove artefacts. We show that UnShadowNet can be easily extended to a fully-supervised set-up to exploit the ground-truth when available. UnShadowNet outperforms existing state-of-the-art approaches on three publicly available shadow datasets (ISTD, adjusted ISTD, SRD) in both the weakly and fully supervised setups.

arXiv commentAccepted for publication at IEEE Access, vol. 11, pp. 87760-87774, 2023

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Representation Learning with Contrastive Predictive Coding
• Learning Representations by Maximizing Mutual Information Across Views
• Direction-aware Spatial Context Features for Shadow Detection and Removal
• Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• Shadow Removal by a Lightness-Guided Network with Training on Unpaired Data
• Learning from Synthetic Shadows for Shadow Detection and Removal
• Physics-based Shadow Image Decomposition for Shadow Removal
• Interactive Removal and Ground Truth for Difficult Shadow Scenes
• Navigating Uncertainty: The Role of Short-Term Trajectory Prediction in Autonomous Vehicle Safety