Noise Robust Generative Adversarial Networks

Abstract

Generative adversarial networks (GANs) are neural networks that learn data distributions through adversarial training. In intensive studies, recent GANs have shown promising results for reproducing training images. However, in spite of noise, they reproduce images with fidelity. As an alternative, we propose a novel family of GANs called noise robust GANs (NR-GANs), which can learn a clean image generator even when training images are noisy. In particular, NR-GANs can solve this problem without having complete noise information (e.g., the noise distribution type, noise amount, or signal-noise relationship). To achieve this, we introduce a noise generator and train it along with a clean image generator. However, without any constraints, there is no incentive to generate an image and noise separately. Therefore, we propose distribution and transformation constraints that encourage the noise generator to capture only the noise-specific components. In particular, considering such constraints under different assumptions, we devise two variants of NR-GANs for signal-independent noise and three variants of NR-GANs for signal-dependent noise. On three benchmark datasets, we demonstrate the effectiveness of NR-GANs in noise robust image generation. Furthermore, we show the applicability of NR-GANs in image denoising. Our code is available at https://github.com/takuhirok/NR-GAN/.

arXiv commentAccepted to CVPR 2020. Project page: https://takuhirok.github.io/NR-GAN/

References in corpus (27)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Neural Discrete Representation Learning
• Self-Attention Generative Adversarial Networks
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Pixel Recurrent Neural Networks
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• NICE: Non-linear Independent Components Estimation
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Learning to Reweight Examples for Robust Deep Learning
• Which Training Methods for GANs do actually Converge?
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• RENOIR - A Dataset for Real Low-Light Image Noise Reduction
• On Convergence and Stability of GANs
• Diversity-Sensitive Conditional Generative Adversarial Networks
• Noise2Self: Blind Denoising by Self-Supervision
• High-Fidelity Image Generation With Fewer Labels
• On Self Modulation for Generative Adversarial Networks
• Robustness of Conditional GANs to Noisy Labels
• Geometric GAN
• Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
• Label-Noise Robust Multi-Domain Image-to-Image Translation