Convergence Rates of Latent Topic Models Under Relaxed Identifiability Conditions

Abstract

In this paper we study the frequentist convergence rate for the Latent Dirichlet Allocation (Blei et al., 2003) topic models. We show that the maximum likelihood estimator converges to one of the finitely many equivalent parameters in Wasserstein's distance metric at a rate of $n^{-1/4}$ without assuming separability or non-degeneracy of the underlying topics and/or the existence of more than three words per document, thus generalizing the previous works of Anandkumar et al. (2012, 2014) from an information-theoretical perspective. We also show that the $n^{-1/4}$ convergence rate is optimal in the worst case.

26 pages, 1 table. Added significantly more expositions, and a numerical procedure to check the order of degeneracy. Proofs slightly altered with explicit constants given at various places

References in corpus (8)

• A Widely Applicable Bayesian Information Criterion
• Learning Topic Models - Going beyond SVD
• Analyzing Tensor Power Method Dynamics in Overcomplete Regime
• Online and Differentially-Private Tensor Decomposition
• Learning Mixtures of Gaussians in High Dimensions
• Polynomial-time Tensor Decompositions with Sum-of-Squares
• Singularity structures and impacts on parameter estimation in finite mixtures of distributions
• Algebraic Problems in Structural Equation Modeling