A Fourier analytique approach to Gaussian mixture learning

Abstract

Suppose that we are given independent, identically distributed random samples $x_1,\cdots ,x_n$ from a mixture at most $k$ many $d$-dimensional spherical Gaussian distributions $\hat{I}{¼}_1,\cdots ,\hat{I}{¼}_{k_0}$ of identical and known variance $\ddot{I}{ƒ}^2$ in each coordinate, such that the minimum ${\ell }^2$ distance between two distinct centers $y_l$ and $y_j$ is greater than $2\hat{I}”\ddot{I}ƒ\min \{\sqrt{d},\sqrt{k}\}$, where $\hat{I}”>C_0$, and $C_0$ is a sufficiently large universal constant. We develop a randomized algorithm that learns the centers $y_l$'s of the Gaussian components to within an ${\ell }^2$ distance of $k^{-{\tilde{C}}_0}$ -- in presence of arbitrarily large number of components and in arbitrary dimension, when the weights are known to be uniform. Furthermore, if the number of components is $k=\hat{I}\text{c◯}(2^d)$, then for arbitrary universal constant $c>0$, even for unknown weights, the algorithm learns the centers to within an ${\ell }^2$ distance of $d^{-{\tilde{C}}_0}$ and the weights up to an accuracy of $c{w}_{min}$, with probability greater than $1-\exp (-k/c)$, provided that the weights lie in $[c/k,1/ck]$, and the minimum separation is just $2c\sqrt{d}$. The number of samples and the computational time is bounded above by $\mathrm{poly}(k,d)$ in either case. Such a bound on the sample and computational complexity was previously unknown in the regime of non-constant dimension, and in particular, when $d$ is not $O(1)$. When $d=O(1)$, this complexity bound follows from work of Regev and Vijayaraghavan, where it has also been shown that the sample complexity of learning a random mixture of Gaussians in a ball of radius $o(\sqrt{d})$ in $d$ dimensions, when $d$ is $\hat{I}˜(\log k)$, is at least super-polynomial in $k,d$, showing that our result is tight in this case.

Almost the same as the published version