A universal framework for learning the elliptical mixture model

Abstract

Mixture modelling using elliptical distributions promises enhanced robustness, flexibility and stability over the widely employed Gaussian mixture model (GMM). However, existing studies based on the elliptical mixture model (EMM) are restricted to several specific types of elliptical probability density functions, which are not supported by general solutions or systematic analysis frameworks; this significantly limits the rigour and the power of EMMs in applications. To this end, we propose a novel general framework for estimating and analysing the EMMs, achieved through Riemannian manifold optimisation. First, we investigate the relationships between Riemannian manifolds and elliptical distributions, and the so established connection between the original manifold and a reformulated one indicates a mismatch between those manifolds, the major cause of failure of the existing optimisation for solving general EMMs. We next propose a universal solver which is based on the optimisation of a re-designed cost and prove the existence of the same optimum as in the original problem; this is achieved in a simple, fast and stable way. We further calculate the influence functions of the EMM as theoretical bounds to quantify robustness to outliers. Comprehensive numerical results demonstrate the ability of the proposed framework to accommodate EMMs with different properties of individual functions in a stable way and with fast convergence speed. Finally, the enhanced robustness and flexibility of the proposed framework over the standard GMM are demonstrated both analytically and through comprehensive simulations.

This work has been accepted to IEEE Transactions on Neural Networks and Learning Systems with DOI:10.1109/TNNLS.2020.3010198. The abstract link is https://ieeexplore.ieee.org/document/9153118

References in corpus (20)

• Manopt, a Matlab toolbox for optimization on manifolds
• Stochastic gradient descent on Riemannian manifolds
• Stochastic Variance Reduction for Nonconvex Optimization
• A Mixture of Generalized Hyperbolic Distributions
• Statistical exponential families: A digest with flash cards
• Regularized $M$-estimators of scatter matrix
• Breakdown points for maximum likelihood estimators of location-scale mixtures
• Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models
• Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds
• Flexible covariance estimation in graphical Gaussian models
• Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences
• Convergence of the EM Algorithm for Gaussian Mixtures with Unbalanced Mixing Coefficients
• Towards Riemannian Accelerated Gradient Methods
• M-Functionals of Multivariate Scatter
• Breakdown Properties of the M-Estimators of Multivariate Scatter
• MixEst: An Estimation Toolbox for Mixture Models
• Geodesic Convexity and Regularized Scatter Estimators
• Estimating densities with nonlinear support using Fisher-Gaussian kernels
• An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization
• Capturing Patterns via Parsimonious t Mixture Models