Learning Topic Models by Belief Propagation
arXiv:1109.3437 · doi:10.1109/TPAMI.2012.185
Abstract
Latent Dirichlet allocation (LDA) is an important hierarchical Bayesian model for probabilistic topic modeling, which attracts worldwide interests and touches on many important applications in text mining, computer vision and computational biology. This paper represents LDA as a factor graph within the Markov random field (MRF) framework, which enables the classic loopy belief propagation (BP) algorithm for approximate inference and parameter estimation. Although two commonly-used approximate inference methods, such as variational Bayes (VB) and collapsed Gibbs sampling (GS), have gained great successes in learning LDA, the proposed BP is competitive in both speed and accuracy as validated by encouraging experimental results on four large-scale document data sets. Furthermore, the BP algorithm has the potential to become a generic learning scheme for variants of LDA-based topic models. To this end, we show how to learn two typical variants of LDA-based topic models, such as author-topic models (ATM) and relational topic models (RTM), using BP based on the factor graph representation.
14 pages, 17 figures
References in corpus (5)
Cited by in corpus (10)
- A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning
- A Topic Modeling Toolbox Using Belief Propagation
- Fast Online EM for Big Topic Modeling
- A New Approach to Speeding Up Topic Modeling
- Communication-Efficient Parallel Belief Propagation for Latent Dirichlet Allocation
- Towards Big Topic Modeling
- Memory-Efficient Topic Modeling
- Residual Belief Propagation for Topic Modeling
- Higher-Order Markov Tag-Topic Models for Tagged Documents and Images
- ALBU: An approximate Loopy Belief message passing algorithm for LDA to improve performance on small data sets