Optimal estimation of Gaussian mixtures via denoised method of moments

Abstract

The Method of Moments [Pea94] is one of the most widely used methods in statistics for parameter estimation, by means of solving the system of equations that match the population and estimated moments. However, in practice and especially for the important case of mixture models, one frequently needs to contend with the difficulties of non-existence or non-uniqueness of statistically meaningful solutions, as well as the high computational cost of solving large polynomial systems. Moreover, theoretical analysis of the method of moments are mainly confined to asymptotic normality style of results established under strong assumptions. This paper considers estimating a $k$-component Gaussian location mixture with a common (possibly unknown) variance parameter. To overcome the aforementioned theoretic and algorithmic hurdles, a crucial step is to denoise the moment estimates by projecting to the truncated moment space (via semidefinite programming) before solving the method of moments equations. Not only does this regularization ensures existence and uniqueness of solutions, it also yields fast solvers by means of Gauss quadrature. Furthermore, by proving new moment comparison theorems in the Wasserstein distance via polynomial interpolation and majorization techniques, we establish the statistical guarantees and adaptive optimality of the proposed procedure, as well as oracle inequality in misspecified models. These results can also be viewed as provable algorithms for Generalized Method of Moments [Han82] which involves non-convex optimization and lacks theoretical guarantees.

References in corpus (5)

• Polynomial Learning of Distribution Families
• Testing composite hypotheses, Hermite polynomials and optimal estimation of a nonsmooth functional
• Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants
• Global analysis of Expectation Maximization for mixtures of two Gaussians
• Chebyshev polynomials, moment matching, and optimal estimation of the unseen

Cited by in corpus (7)

• Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures
• Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models
• Optimal Bayesian estimation of Gaussian mixtures with growing number of components
• Asymptotics of smoothed Wasserstein distances
• The EM Algorithm is Adaptively-Optimal for Unbalanced Symmetric Gaussian Mixtures
• Uniform Convergence Rates for Maximum Likelihood Estimation under Two-Component Gaussian Mixture Models
• On the Competitive Analysis and High Accuracy Optimality of Profile Maximum Likelihood