Tessellated Wasserstein Auto-Encoders

Abstract

Non-adversarial generative models such as variational auto-encoder (VAE), Wasserstein auto-encoders with maximum mean discrepancy (WAE-MMD), sliced-Wasserstein auto-encoder (SWAE) are relatively easy to train and have less mode collapse compared to Wasserstein auto-encoder with generative adversarial network (WAE-GAN). However, they are not very accurate in approximating the target distribution in the latent space because they don't have a discriminator to detect the minor difference between real and fake. To this end, we develop a novel non-adversarial framework called Tessellated Wasserstein Auto-encoders (TWAE) to tessellate the support of the target distribution into a given number of regions by the centroidal Voronoi tessellation (CVT) technique and design batches of data according to the tessellation instead of random shuffling for accurate computation of discrepancy. Theoretically, we demonstrate that the error of estimate to the discrepancy decreases when the numbers of samples $n$ and regions $m$ of the tessellation become larger with rates of $\mathcal{O}(\frac{1}{\sqrt{n}})$ and $\mathcal{O}(\frac{1}{\sqrt{m}})$, respectively. Given fixed $n$ and $m$, a necessary condition for the upper bound of measurement error to be minimized is that the tessellation is the one determined by CVT. TWAE is very flexible to different non-adversarial metrics and can substantially enhance their generative performance in terms of Fréchet inception distance (FID) compared to VAE, WAE-MMD, SWAE. Moreover, numerical results indeed demonstrate that TWAE is competitive to the adversarial model WAE-GAN, demonstrating its powerful generative ability.

arXiv comment38 pages, 8 figures

References in corpus (16)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• NIPS 2016 Tutorial: Generative Adversarial Networks
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• The sphere packing problem in dimension 24
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Wasserstein Auto-Encoders
• Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm
• Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
• On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms
• Reducing Noise in GAN Training with Variance Reduced Extragradient
• Subspace Robust Wasserstein Distances
• Learning Implicit Generative Models with Theoretical Guarantees

Cited by in corpus (1)

• Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Space