A Geometric Analysis of Initialization Bias in Spherical $K$-means in the Weak Signal Regime

Abstract

We study initialization bias in spherical $K$-means for weakly informative directional mixtures. We model the observations by a $K$-component von Mises-Fisher mixture with a small concentration parameter $\kappa$, corresponding to a high-dispersion regime in which the data provide limited information about the underlying directions. Our analysis begins with the limiting case $\kappa =0$ (corresponding to a uniform distribution over the sphere), where one population spherical $K$-means update is governed entirely by the Voronoi tessellation induced by the initialized templates. For uniformly random initializations in fixed dimension $d$, the updated templates become asymptotically aligned with their initial values as $K\to \infty$: the average squared geodesic error scales as $O(K^{-2/(d-1)})$, while the worst-case error is $O((\log K/K{)}^{2/(d-1)})$. We then show that, in the weak-signal regime of small positive $\kappa$, the population update remains an $O(\kappa )$ perturbation of this limiting map. Thus, in the weak-signal regime, spherical $K$-means can preserve initialization-induced structure despite the presence of a genuine but highly dispersed directional signal.