A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data

Abstract

This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A classical imputation method, the Expectation Maximization (EM) algorithm for Gaussian mixture models, has shown interesting properties when compared to other popular approaches such as those based on k-nearest neighbors or on multiple imputations by chained equations. However, Gaussian mixture models are known to be non-robust to heterogeneous data, which can lead to poor estimation performance when the data is contaminated by outliers or follows non-Gaussian distributions. To overcome this issue, a new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data. This paper shows that this problem reduces to the estimation of a mixture of Angular Gaussian distributions under generic assumptions (i.e., each sample is drawn from a mixture of elliptical distributions, which is possibly different for one sample to another). In that case, the complete-data likelihood associated with mixtures of elliptical distributions is well adapted to the EM framework with missing data thanks to its conditional distribution, which is shown to be a multivariate $t$-distribution. Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data. Furthermore, experiments conducted on real-world datasets show that this algorithm is very competitive when compared to other classical imputation methods.

References in corpus (14)

• Scikit-learn: Machine Learning in Python
• MissForest - nonparametric missing value imputation for mixed-type data
• High-Dimensional Data Clustering
• A Mixture of Generalized Hyperbolic Distributions
• Parameter Estimation For Multivariate Generalized Gaussian Distributions
• Parameter Estimation of Heavy-Tailed AR Model with Missing Data via Stochastic EM
• Structured Covariance Matrix Estimation with Missing-Data for Radar Applications via Expectation-Maximization
• Mixtures of Generalized Hyperbolic Distributions and Mixtures of Skew-t Distributions for Model-Based Clustering with Incomplete Data
• Outlier detection at the parcel-level in wheat and rapeseed crops using multispectral and SAR time series
• Reconstruction of Sentinel-2 Time Series Using Robust Gaussian Mixture Models -- Application to the Detection of Anomalous Crop Development in wheat and rapeseed crops
• Provable Subspace Tracking from Missing Data and Matrix Completion
• Efficient EM Training of Gaussian Mixtures with Missing Data
• A flexible EM-like clustering algorithm for noisy data
• Model-based Clustering with Missing Not At Random Data

Cited by in corpus (1)

• Signal Recovery Using a Spiked Mixture Model