Trust the Critics: Generatorless and Multipurpose WGANs with Initial Convergence Guarantees

Abstract

Inspired by ideas from optimal transport theory we present Trust the Critics (TTC), a new algorithm for generative modelling. This algorithm eliminates the trainable generator from a Wasserstein GAN; instead, it iteratively modifies the source data using gradient descent on a sequence of trained critic networks. This is motivated in part by the misalignment which we observed between the optimal transport directions provided by the gradients of the critic and the directions in which data points actually move when parametrized by a trainable generator. Previous work has arrived at similar ideas from different viewpoints, but our basis in optimal transport theory motivates the choice of an adaptive step size which greatly accelerates convergence compared to a constant step size. Using this step size rule, we prove an initial geometric convergence rate in the case of source distributions with densities. These convergence rates cease to apply only when a non-negligible set of generated data is essentially indistinguishable from real data. Resolving the misalignment issue improves performance, which we demonstrate in experiments that show that given a fixed number of training epochs, TTC produces higher quality images than a comparable WGAN, albeit at increased memory requirements. In addition, TTC provides an iterative formula for the transformed density, which traditional WGANs do not. Finally, TTC can be applied to map any source distribution onto any target; we demonstrate through experiments that TTC can obtain competitive performance in image generation, translation, and denoising without dedicated algorithms.

arXiv comment20 pages, 8 figures

References in corpus (13)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• On the regularization of Wasserstein GANs
• FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
• How to train your neural ODE: the world of Jacobian and kinetic regularization
• Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
• On the Convergence and Robustness of Training GANs with Regularized Optimal Transport
• End-to-end reconstruction meets data-driven regularization for inverse problems
• Composite Functional Gradient Learning of Generative Adversarial Models
• Wasserstein GANs with Gradient Penalty Compute Congested Transport