Correction by Projection: Denoising Images with Generative Adversarial Networks

Abstract

Generative adversarial networks (GANs) transform low-dimensional latent vectors into visually plausible images. If the real dataset contains only clean images, then ostensibly, the manifold learned by the GAN should contain only clean images. In this paper, we propose to denoise corrupted images by finding the nearest point on the GAN manifold, recovering latent vectors by minimizing distances in image space. We first demonstrate that given a corrupted version of an image that truly lies on the GAN manifold, we can approximately recover the latent vector and denoise the image, obtaining significantly higher quality, comparing with BM3D. Next, we demonstrate that latent vectors recovered from noisy images exhibit a consistent bias. By subtracting this bias before projecting back to image space, we improve denoising results even further. Finally, even for unseen images, our method performs better at denoising better than BM3D. Notably, the basic version of our method (without bias correction) requires no prior knowledge on the noise variance. To achieve the highest possible denoising quality, the best performing signal processing based methods, such as BM3D, require an estimate of the blur kernel.

References in corpus (2)

• Inverting The Generator Of A Generative Adversarial Network (II)
• Learning to Avoid Errors in GANs by Manipulating Input Spaces

Cited by in corpus (14)

• Blockchain-based Digital Twins: Research Trends, Issues, and Future Challenges
• Compressed Sensing with Deep Image Prior and Learned Regularization
• A survey of synthetic data augmentation methods in computer vision
• Digital Twin: Values, Challenges and Enablers
• Deep Learning on Image Denoising: An overview
• Fast and Provable ADMM for Learning with Generative Priors
• Deep Learning Techniques for Inverse Problems in Imaging
• Deep Learning for Image Denoising: A Survey
• Compressed Sensing MRI Reconstruction Regularized by VAEs with Structured Image Covariance
• Inverting Deep Generative models, One layer at a time
• Inference in Multi-Layer Networks with Matrix-Valued Unknowns
• ReLGAN: Generalization of Consistency for GAN with Disjoint Constraints and Relative Learning of Generative Processes for Multiple Transformation Learning
• An approach to image denoising using manifold approximation without clean images
• Controllable Confidence-Based Image Denoising