Unsupervised Portrait Shadow Removal via Generative Priors

Abstract

Portrait images often suffer from undesirable shadows cast by casual objects or even the face itself. While existing methods for portrait shadow removal require training on a large-scale synthetic dataset, we propose the first unsupervised method for portrait shadow removal without any training data. Our key idea is to leverage the generative facial priors embedded in the off-the-shelf pretrained StyleGAN2. To achieve this, we formulate the shadow removal task as a layer decomposition problem: a shadowed portrait image is constructed by the blending of a shadow image and a shadow-free image. We propose an effective progressive optimization algorithm to learn the decomposition process. Our approach can also be extended to portrait tattoo removal and watermark removal. Qualitative and quantitative experiments on a real-world portrait shadow dataset demonstrate that our approach achieves comparable performance with supervised shadow removal methods. Our source code is available at https://github.com/YingqingHe/Shadow-Removal-via-Generative-Priors.

Accepted to ACM MM 2021 (Oral). Code is available at: https://github.com/YingqingHe/Shadow-Removal-via-Generative-Priors

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Direction-aware Spatial Context Features for Shadow Detection
• Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data
• Photographic Image Synthesis with Cascaded Refinement Networks
• ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal
• Shadow Removal via Shadow Image Decomposition

Cited by in corpus (2)

• Language-Driven Interactive Shadow Detection
• Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era