Implicit Manifold Learning on Generative Adversarial Networks

Abstract

This paper raises an implicit manifold learning perspective in Generative Adversarial Networks (GANs), by studying how the support of the learned distribution, modelled as a submanifold ${\mathcal{M}}_{\theta }$, perfectly match with ${\mathcal{M}}_r$, the support of the real data distribution. We show that optimizing Jensen-Shannon divergence forces ${\mathcal{M}}_{\theta }$ to perfectly match with ${\mathcal{M}}_r$, while optimizing Wasserstein distance does not. On the other hand, by comparing the gradients of the Jensen-Shannon divergence and the Wasserstein distances ($W_1$ and $W_2^2$) in their primal forms, we conjecture that Wasserstein $W_2^2$ may enjoy desirable properties such as reduced mode collapse. It is therefore interesting to design new distances that inherit the best from both distances.

References in corpus (8)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Do GANs actually learn the distribution? An empirical study

Cited by in corpus (1)

• Adversarial Computation of Optimal Transport Maps