Feature Quantization Improves GAN Training

Abstract

The instability in GAN training has been a long-standing problem despite remarkable research efforts. We identify that instability issues stem from difficulties of performing feature matching with mini-batch statistics, due to a fragile balance between the fixed target distribution and the progressively generated distribution. In this work, we propose Feature Quantization (FQ) for the discriminator, to embed both true and fake data samples into a shared discrete space. The quantized values of FQ are constructed as an evolving dictionary, which is consistent with feature statistics of the recent distribution history. Hence, FQ implicitly enables robust feature matching in a compact space. Our method can be easily plugged into existing GAN models, with little computational overhead in training. We apply FQ to 3 representative GAN models on 9 benchmarks: BigGAN for image generation, StyleGAN for face synthesis, and U-GAT-IT for unsupervised image-to-image translation. Extensive experimental results show that the proposed FQ-GAN can improve the FID scores of baseline methods by a large margin on a variety of tasks, achieving new state-of-the-art performance.

The first two authors contributed equally to this manuscript. ICML 2020. Code: https://github.com/YangNaruto/FQ-GAN

References in corpus (10)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• Neural Discrete Representation Learning
• Which Training Methods for GANs do actually Converge?
• U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
• On Convergence and Stability of GANs
• Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
• On Self Modulation for Generative Adversarial Networks
• Generative Adversarial Network Training is a Continual Learning Problem
• Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks

Cited by in corpus (10)

• Training Generative Adversarial Networks with Limited Data
• Conditional GANs with Auxiliary Discriminative Classifier
• Self-Supervised GANs with Label Augmentation
• Styleformer: Transformer based Generative Adversarial Networks with Style Vector
• Scalable Balanced Training of Conditional Generative Adversarial Neural Networks on Image Data
• Unpaired Image-to-Image Translation via Latent Energy Transport
• Omni-GAN: On the Secrets of cGANs and Beyond
• not-so-BigGAN: Generating High-Fidelity Images on Small Compute with Wavelet-based Super-Resolution
• Instance Selection for GANs
• Enhancing Stability in Training Conditional Generative Adversarial Networks via Selective Data Matching