Blur, Noise, and Compression Robust Generative Adversarial Networks

Abstract

Generative adversarial networks (GANs) have gained considerable attention owing to their ability to reproduce images. However, they can recreate training images faithfully despite image degradation in the form of blur, noise, and compression, generating similarly degraded images. To solve this problem, the recently proposed noise robust GAN (NR-GAN) provides a partial solution by demonstrating the ability to learn a clean image generator directly from noisy images using a two-generator model comprising image and noise generators. However, its application is limited to noise, which is relatively easy to decompose owing to its additive and reversible characteristics, and its application to irreversible image degradation, in the form of blur, compression, and combination of all, remains a challenge. To address these problems, we propose blur, noise, and compression robust GAN (BNCR-GAN) that can learn a clean image generator directly from degraded images without knowledge of degradation parameters (e.g., blur kernel types, noise amounts, or quality factor values). Inspired by NR-GAN, BNCR-GAN uses a multiple-generator model composed of image, blur-kernel, noise, and quality-factor generators. However, in contrast to NR-GAN, to address irreversible characteristics, we introduce masking architectures adjusting degradation strength values in a data-driven manner using bypasses before and after degradation. Furthermore, to suppress uncertainty caused by the combination of blur, noise, and compression, we introduce adaptive consistency losses imposing consistency between irreversible degradation processes according to the degradation strengths. We demonstrate the effectiveness of BNCR-GAN through large-scale comparative studies on CIFAR-10 and a generality analysis on FFHQ. In addition, we demonstrate the applicability of BNCR-GAN in image restoration.

Accepted to CVPR 2021. Project page: https://takuhirok.github.io/BNCR-GAN/

References in corpus (18)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Neural Discrete Representation Learning
• Self-Attention Generative Adversarial Networks
• NICE: Non-linear Independent Components Estimation
• Which Training Methods for GANs do actually Converge?
• GAIN: Missing Data Imputation using Generative Adversarial Nets
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Diversity-Sensitive Conditional Generative Adversarial Networks
• Noise2Self: Blind Denoising by Self-Supervision
• On Self Modulation for Generative Adversarial Networks
• Robustness of Conditional GANs to Noisy Labels
• MisGAN: Learning from Incomplete Data with Generative Adversarial Networks
• Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
• Training Image Estimators without Image Ground-Truth
• Label-Noise Robust Multi-Domain Image-to-Image Translation