Radon Sobolev Variational Auto-Encoders
arXiv:1911.13135
Abstract
The quality of generative models (such as Generative adversarial networks and Variational Auto-Encoders) depends heavily on the choice of a good probability distance. However some popular metrics like the Wasserstein or the Sliced Wasserstein distances, the Jensen-Shannon divergence, the Kullback-Leibler divergence, lack convenient properties such as (geodesic) convexity, fast evaluation and so on. To address these shortcomings, we introduce a class of distances that have built-in convexity. We investigate the relationship with some known paradigms (sliced distances - a synonym for Radon distances -, reproducing kernel Hilbert spaces, energy distances). The distances are shown to possess fast implementations and are included in an adapted Variational Auto-Encoder termed Radon Sobolev Variational Auto-Encoder (RS-VAE) which produces high quality results on standard generative datasets. Keywords: Variational Auto-Encoder; Generative model; Sobolev spaces; Radon Sobolev Variational Auto-Encoder;
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Equivalence of distance-based and RKHS-based statistics in hypothesis testing
- The Cramer Distance as a Solution to Biased Wasserstein Gradients
- Wasserstein Auto-Encoders
- Cramer-Wold AutoEncoder
- Characterisation of homogeneous fractional Sobolev spaces