Solving general elliptical mixture models through an approximate Wasserstein manifold

Abstract

We address the estimation problem for general finite mixture models, with a particular focus on the elliptical mixture models (EMMs). Compared to the widely adopted Kullback-Leibler divergence, we show that the Wasserstein distance provides a more desirable optimisation space. We thus provide a stable solution to the EMMs that is both robust to initialisations and reaches a superior optimum by adaptively optimising along a manifold of an approximate Wasserstein distance. To this end, we first provide a unifying account of computable and identifiable EMMs, which serves as a basis to rigorously address the underpinning optimisation problem. Due to a probability constraint, solving this problem is extremely cumbersome and unstable, especially under the Wasserstein distance. To relieve this issue, we introduce an efficient optimisation method on a statistical manifold defined under an approximate Wasserstein distance, which allows for explicit metrics and computable operations, thus significantly stabilising and improving the EMM estimation. We further propose an adaptive method to accelerate the convergence. Experimental results demonstrate the excellent performance of the proposed EMM solver.

This work has been accepted to AAAI2020. Note that this version also corrects a small error on the Equation (16) in proof

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Wasserstein GAN
• New insights and perspectives on the natural gradient method
• A Mixture of Generalized Hyperbolic Distributions
• Riemannian Adaptive Optimization Methods
• Conic geometric optimisation on the manifold of positive definite matrices
• Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models
• Global analysis of Expectation Maximization for mixtures of two Gaussians
• Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions
• Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences
• First-order Methods for Geodesically Convex Optimization
• Convergence of the EM Algorithm for Gaussian Mixtures with Unbalanced Mixing Coefficients
• A numerical algorithm for $L_2$ semi-discrete optimal transport in 3D

Cited by in corpus (1)

• Reciprocal Adversarial Learning via Characteristic Functions