A Two-round Variant of EM for Gaussian Mixtures

Abstract

Given a set of possible models (e.g., Bayesian network structures) and a data sample, in the unsupervised model selection problem the task is to choose the most accurate model with respect to the domain joint probability distribution. In contrast to this, in supervised model selection it is a priori known that the chosen model will be used in the future for prediction tasks involving more ``focused' predictive distributions. Although focused predictive distributions can be produced from the joint probability distribution by marginalization, in practice the best model in the unsupervised sense does not necessarily perform well in supervised domains. In particular, the standard marginal likelihood score is a criterion for the unsupervised task, and, although frequently used for supervised model selection also, does not perform well in such tasks. In this paper we study the performance of the marginal likelihood score empirically in supervised Bayesian network selection tasks by using a large number of publicly available classification data sets, and compare the results to those obtained by alternative model selection criteria, including empirical crossvalidation methods, an approximation of a supervised marginal likelihood measure, and a supervised version of Dawids prequential(predictive sequential) principle.The results demonstrate that the marginal likelihood score does NOT perform well FOR supervised model selection, WHILE the best results are obtained BY using Dawids prequential r napproach.

Appears in Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI2000)

Cited by in corpus (36)

• Polynomial Learning of Distribution Families
• Learning mixtures of separated nonspherical Gaussians
• Learning mixtures of structured distributions over discrete domains
• Settling the Polynomial Learnability of Mixtures of Gaussians
• High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality
• The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures
• HIRL: Hierarchical Inverse Reinforcement Learning for Long-Horizon Tasks with Delayed Rewards
• Learning Mixtures of Gaussians using the k-means Algorithm
• On Bayes Risk Lower Bounds
• Training Gaussian Mixture Models at Scale via Coresets
• Learning Gaussian Mixtures with Arbitrary Separation
• Curse of Heterogeneity: Computational Barriers in Sparse Mixture Models and Phase Retrieval
• On some provably correct cases of variational inference for topic models
• Separating populations with wide data: A spectral analysis
• The Dynamic ECME Algorithm
• Algorithmic Foundations for the Diffraction Limit
• Settling the Robust Learnability of Mixtures of Gaussians
• Learning Mixtures of Bernoulli Templates by Two-Round EM with Performance Guarantee
• PAC Learning Mixtures of Axis-Aligned Gaussians with No Separation Assumption
• TRUST-TECH based Methods for Optimization and Learning
• Mapping Energy Landscapes of Non-Convex Learning Problems
• Support Localization and the Fisher Metric for off-the-grid Sparse Regularization
• Convergence Analysis of Gradient EM for Multi-component Gaussian Mixture
• Super-Resolution Off the Grid
• Provably Efficient Exploration for Reinforcement Learning Using Unsupervised Learning
• The Informativeness of K -Means for Learning Mixture Models
• Lower Bounds on the Total Variation Distance Between Mixtures of Two Gaussians
• On Convergence of Epanechnikov Mean Shift
• A concentration theorem for projections
• Integer Feasibility of Random Polytopes
• Linear Time Clustering for High Dimensional Mixtures of Gaussian Clouds
• Algorithms for finding $k$ in $k$-means
• Unsupervised Learning of GMM with a Uniform Background Component
• Geometric Methods for Robust Data Analysis in High Dimension
• Comparing EM with GD in Mixture Models of Two Components
• Learning Balanced Mixtures of Discrete Distributions with Small Sample