On the Optimization Landscape of Maximum Mean Discrepancy

Abstract

Generative models have been successfully used for generating realistic signals. Because the likelihood function is typically intractable in most of these models, the common practice is to use "implicit" models that avoid likelihood calculation. However, it is hard to obtain theoretical guarantees for such models. In particular, it is not understood when they can globally optimize their non-convex objectives. Here we provide such an analysis for the case of Maximum Mean Discrepancy (MMD) learning of generative models. We prove several optimality results, including for a Gaussian distribution with low rank covariance (where likelihood is inapplicable) and a mixture of Gaussians. Our analysis shows that that the MMD optimization landscape is benign in these cases, and therefore gradient based methods will globally minimize the MMD objective.

References in corpus (8)

• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Generative Moment Matching Networks
• Demystifying MMD GANs
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
• When Are Nonconvex Problems Not Scary?
• Maximum Mean Discrepancy Gradient Flow
• On the Limitations of First-Order Approximation in GAN Dynamics
• Understanding GANs: the LQG Setting