Direction-aware Spatial Context Features for Shadow Detection and Removal

Abstract

Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the spatial image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features at different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a Euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistencies in the training pairs. We employed two shadow detection benchmark datasets and two shadow removal benchmark datasets, and performed various experiments to evaluate our method. Experimental results show that our method performs favorably against the state-of-the-art methods for both shadow detection and shadow removal.

Accepted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). This is the journal version of arXiv:1712.04142, which was accepted for oral presentation in CVPR 2018

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• Direction-aware Spatial Context Features for Shadow Detection

Cited by in corpus (25)

• A Survey of Deep Learning-based Object Detection
• Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data
• Shadow Removal by a Lightness-Guided Network with Training on Unpaired Data
• Revisiting Shadow Detection: A New Benchmark Dataset for Complex World
• SAC-Net: Spatial Attenuation Context for Salient Object Detection
• Learning from Synthetic Shadows for Shadow Detection and Removal
• Physics-based Shadow Image Decomposition for Shadow Removal
• Local Label Point Correction for Edge Detection of Overlapping Cervical Cells
• Instance Shadow Detection with A Single-Stage Detector
• Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN
• RIS-GAN: Explore Residual and Illumination with Generative Adversarial Networks for Shadow Removal
• Context-Aware Mutual Learning for Blind Image Inpainting and Beyond
• Self-Supervised Shadow Removal
• UnShadowNet: Illumination Critic Guided Contrastive Learning For Shadow Removal
• Auto-Exposure Fusion for Single-Image Shadow Removal
• Instance Shadow Detection
• FieldNet: Efficient Real-Time Shadow Removal for Enhanced Vision in Field Robotics
• Wavelet-Based Dual-Branch Network for Image Demoireing
• Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era
• CANet: A Context-Aware Network for Shadow Removal
• From Shadow Generation to Shadow Removal
• Triple-cooperative Video Shadow Detection
• Portrait Shadow Manipulation
• No Shadow Left Behind: Removing Objects and their Shadows using Approximate Lighting and Geometry
• Blind Image Decomposition