SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains
arXiv:2012.05535
Abstract
This paper observes that there is an issue of high frequencies missing in the discriminator of standard GAN, and we reveal it stems from downsampling layers employed in the network architecture. This issue makes the generator lack the incentive from the discriminator to learn high-frequency content of data, resulting in a significant spectrum discrepancy between generated images and real images. Since the Fourier transform is a bijective mapping, we argue that reducing this spectrum discrepancy would boost the performance of GANs. To this end, we introduce SSD-GAN, an enhancement of GANs to alleviate the spectral information loss in the discriminator. Specifically, we propose to embed a frequency-aware classifier into the discriminator to measure the realness of the input in both the spatial and spectral domains. With the enhanced discriminator, the generator of SSD-GAN is encouraged to learn high-frequency content of real data and generate exact details. The proposed method is general and can be easily integrated into most existing GANs framework without excessive cost. The effectiveness of SSD-GAN is validated on various network architectures, objective functions, and datasets. Code will be available at https://github.com/cyq373/SSD-GAN.
arXiv commentAccepted to AAAI 2021. Code: https://github.com/cyq373/SSD-GAN
References in corpus (8)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- Leveraging Frequency Analysis for Deep Fake Image Recognition
- On the Spectral Bias of Neural Networks
- Diversity-Sensitive Conditional Generative Adversarial Networks
- Fourier Spectrum Discrepancies in Deep Network Generated Images
- Band-limited Training and Inference for Convolutional Neural Networks
- Mimicry: Towards the Reproducibility of GAN Research
Cited by in corpus (5)
- A Systematic Survey of Regularization and Normalization in GANs
- UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution
- SWAGAN: A Style-based Wavelet-driven Generative Model
- Unsupervised Image Denoising with Frequency Domain Knowledge
- F-Drop&Match: GANs with a Dead Zone in the High-Frequency Domain