Scalable Balanced Training of Conditional Generative Adversarial Neural Networks on Image Data
arXiv:2102.10485 · doi:10.1007/s11227-021-03808-2
Abstract
We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1,000 processes and 2,000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.
References in corpus (28)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Conditional Generative Adversarial Nets
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- An overview of gradient descent optimization algorithms
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Self-Attention Generative Adversarial Networks
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Going Deeper with Convolutions
- Improved Techniques for Training GANs
- NIPS 2016 Tutorial: Generative Adversarial Networks
- cGANs with Projection Discriminator
- How to Construct Deep Recurrent Neural Networks
- A Note on the Inception Score
- Equilibrated adaptive learning rates for non-convex optimization
- Diversity-Sensitive Conditional Generative Adversarial Networks
- Differentiable Augmentation for Data-Efficient GAN Training
- Are GANs Created Equal? A Large-Scale Study
- Competitive Gradient Descent
- AttGAN: Facial Attribute Editing by Only Changing What You Want
- Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
- Feature Quantization Improves GAN Training
- Multiagent Rollout Algorithms and Reinforcement Learning
- cGANs with Multi-Hinge Loss
- Generative Adversarial Network based on Resnet for Conditional Image Restoration
- Multi-Agent Decentralized Network Interdiction Games