Cramer-Wold AutoEncoder

Abstract

We propose a new generative model, Cramer-Wold Autoencoder (CWAE). Following WAE, we directly encourage normality of the latent space. Our paper uses also the recent idea from Sliced WAE (SWAE) model, which uses one-dimensional projections as a method of verifying closeness of two distributions. The crucial new ingredient is the introduction of a new (Cramer-Wold) metric in the space of densities, which replaces the Wasserstein metric used in SWAE. We show that the Cramer-Wold metric between Gaussian mixtures is given by a simple analytic formula, which results in the removal of sampling necessary to estimate the cost function in WAE and SWAE models. As a consequence, while drastically simplifying the optimization procedure, CWAE produces samples of a matching perceptual quality to other SOTA models.

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Wasserstein Auto-Encoders
• Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model

Cited by in corpus (8)

• Distributional Sliced-Wasserstein and Applications to Generative Modeling
• HyperFlow: Representing 3D Objects as Surfaces
• Sliced Iterative Normalizing Flows
• Radon Sobolev Variational Auto-Encoders
• "Best-of-Many-Samples" Distribution Matching
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
• Toroidal AutoEncoder
• One-element Batch Training by Moving Window