Sparse Stochastic Inference for Latent Dirichlet allocation

Abstract

We present a hybrid algorithm for Bayesian topic models that combines the efficiency of sparse Gibbs sampling with the scalability of online stochastic inference. We used our algorithm to analyze a corpus of 1.2 million books (33 billion words) with thousands of topics. Our approach reduces the bias of variational inference and generalizes to many Bayesian hidden-variable models.

Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012)

References in corpus (1)

• Online EM Algorithm for Latent Data Models

Cited by in corpus (23)

• Structured Stochastic Variational Inference
• Copula variational inference
• Stochastic Variational Inference
• Online Bayesian Passive-Aggressive Learning
• Fast Online EM for Big Topic Modeling
• Topic Discovery in Massive Text Corpora Based on Min-Hashing
• A New Approach to Speeding Up Topic Modeling
• Fast Learning of Clusters and Topics via Sparse Posteriors
• Guaranteed inference in topic models
• Scalable and Robust Construction of Topical Hierarchies
• Towards Big Topic Modeling
• Scalable Probabilistic Entity-Topic Modeling
• Sampled Weighted Min-Hashing for Large-Scale Topic Mining
• Managing sparsity, time, and quality of inference in topic models
• A Nonparametric Bayesian Approach Toward Stacked Convolutional Independent Component Analysis
• An Empirical Study of Stochastic Variational Algorithms for the Beta Bernoulli Process
• Inference in topic models: sparsity and trade-off
• Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation
• Fast Sampling for Bayesian Max-Margin Models
• Statistical inference on errorfully observed graphs
• Online Inference for Relation Extraction with a Reduced Feature Set
• Online Bayesian Collaborative Topic Regression
• Efficient Correlated Topic Modeling with Topic Embedding