2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs

Abstract

We provide a framework to approximate the 2-Wasserstein distance and the optimal transport map, amenable to efficient training as well as statistical and geometric analysis. With the quadratic cost and considering the Kantorovich dual form of the optimal transportation problem, the Brenier theorem states that the optimal potential function is convex and the optimal transport map is the gradient of the optimal potential function. Using this geometric structure, we restrict the optimization problem to different parametrized classes of convex functions and pay special attention to the class of input-convex neural networks. We analyze the statistical generalization and the discriminative power of the resulting approximate metric, and we prove a restricted moment-matching property for the approximate optimal map. Finally, we discuss a numerical algorithm to solve the restricted optimization problem and provide numerical experiments to illustrate and compare the proposed approach with the established regularization-based approaches. We further discuss practical implications of our proposal in a modular and interpretable design for GANs which connects the generator training with discriminator computations to allow for learning an overall composite generator.

References in corpus (28)

• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Improved Training of Wasserstein GANs
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Bayesian Inference with Optimal Maps
• Input Convex Neural Networks
• Smooth and Sparse Optimal Transport
• Joint Distribution Optimal Transportation for Domain Adaptation
• Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm
• Approximation and Convergence Properties of Generative Adversarial Learning
• On the Discrimination-Generalization Tradeoff in GANs
• Sparse/Robust Estimation and Kalman Smoothing with Nonsmooth Log-Concave Densities: Modeling, Computation, and Theory
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• Improving GANs Using Optimal Transport
• Understanding GANs: the LQG Setting
• Approximability of Discriminators Implies Diversity in GANs
• Gibbs flow for approximate transport with applications to Bayesian computation
• Mode Collapse and Regularity of Optimal Transportation Maps
• Local moment matching: A unified methodology for symmetric functional estimation and distribution estimation under Wasserstein distance
• A General Duality Theorem for the Monge--Kantorovich Transport Problem

Cited by in corpus (9)

• Optimal transport mapping via input convex neural networks
• GANs May Have No Nash Equilibria
• Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks
• Wasserstein-2 Generative Networks
• Continuous Regularized Wasserstein Barycenters
• Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
• Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
• Generative Modeling with Optimal Transport Maps
• Physics Informed Convex Artificial Neural Networks (PICANNs) for Optimal Transport based Density Estimation