Exploring The Effect of High-frequency Components in GANs Training

Abstract

Generative Adversarial Networks (GANs) have the ability to generate images that are visually indistinguishable from real images. However, recent studies have revealed that generated and real images share significant differences in the frequency domain. In this paper, we explore the effect of high-frequency components in GANs training. According to our observation, during the training of most GANs, severe high-frequency differences make the discriminator focus on high-frequency components excessively, which hinders the generator from fitting the low-frequency components that are important for learning images' content. Then, we propose two simple yet effective frequency operations for eliminating the side effects caused by high-frequency differences in GANs training: High-Frequency Confusion (HFC) and High-Frequency Filter (HFF). The proposed operations are general and can be applied to most existing GANs with a fraction of the cost. The advanced performance of the proposed operations is verified on multiple loss functions, network architectures, and datasets. Specifically, the proposed HFF achieves significant improvements of $42.5\%$ FID on CelebA (128*128) unconditional generation based on SNGAN, $30.2\%$ FID on CelebA unconditional generation based on SSGAN, and $69.3\%$ FID on CelebA unconditional generation based on InfoMAXGAN.

References in corpus (12)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• On the Spectral Bias of Neural Networks
• PEPSI++: Fast and Lightweight Network for Image Inpainting
• The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies
• Towards Understanding the Spectral Bias of Deep Learning
• Understanding training and generalization in deep learning by Fourier analysis
• Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
• Frequency Principle in Deep Learning with General Loss Functions and Its Potential Application
• F-Drop&Match: GANs with a Dead Zone in the High-Frequency Domain