Lipizzaner: A System That Scales Robust Generative Adversarial Network Training
arXiv:1811.12843
Abstract
GANs are difficult to train due to convergence pathologies such as mode and discriminator collapse. We introduce Lipizzaner, an open source software system that allows machine learning engineers to train GANs in a distributed and robust way. Lipizzaner distributes a competitive coevolutionary algorithm which, by virtue of dual, adapting, generator and discriminator populations, is robust to collapses. The algorithm is well suited to efficient distribution because it uses a spatial grid abstraction. Training is local to each cell and strong intermediate training results are exchanged among overlapping neighborhoods allowing high performing solutions to propagate and improve with more rounds of training. Experiments on common image datasets overcome critical collapses. Communication overhead scales linearly when increasing the number of compute instances and we observe that increasing scale leads to improved model performance.
arXiv commentSystems for ML Workshop (MLSYS) at NeurIPS 2018
References in corpus (8)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Towards Principled Methods for Training Generative Adversarial Networks
- Do GANs actually learn the distribution? An empirical study
- On the Limitations of First-Order Approximation in GAN Dynamics
- Ensembles of Generative Adversarial Networks
- Distributional Adversarial Networks