Nested Hierarchical Dirichlet Processes

Abstract

We develop a nested hierarchical Dirichlet process (nHDP) for hierarchical topic modeling. The nHDP is a generalization of the nested Chinese restaurant process (nCRP) that allows each word to follow its own path to a topic node according to a document-specific distribution on a shared tree. This alleviates the rigid, single-path formulation of the nCRP, allowing a document to more easily express thematic borrowings as a random effect. We derive a stochastic variational inference algorithm for the model, in addition to a greedy subtree selection method for each document, which allows for efficient inference using massive collections of text documents. We demonstrate our algorithm on 1.8 million documents from The New York Times and 3.3 million documents from Wikipedia.

To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence, Special Issue on Bayesian Nonparametrics

Cited by in corpus (40)

• A network approach to topic models
• WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling
• Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC
• Stochastic Variational Inference
• Dirichlet belief networks for topic structure learning
• Nonparametric Spherical Topic Modeling with Word Embeddings
• Unsupervised Terminological Ontology Learning based on Hierarchical Topic Modeling
• Expressive recommender systems through normalized nonnegative models
• Nonparametric Bayesian Negative Binomial Factor Analysis
• An Analysis of Lemmatization on Topic Models of Morphologically Rich Language
• Mass-structure of weighted real trees
• Neural Topic Model via Optimal Transport
• Recommender systems inspired by the structure of quantum theory
• Correlated Random Measures
• Nonparametric Bayesian Two-Level Clustering for Subject-Level Single-Cell Expression Data
• A Nested HDP for Hierarchical Topic Models
• Bayesian Transfer Learning: An Overview of Probabilistic Graphical Models for Transfer Learning
• Latent Tree Models for Hierarchical Topic Detection
• Bayesian Nonparametric Models for Multiple Raters: a General Statistical Framework
• Posterior Distribution for the Number of Clusters in Dirichlet Process Mixture Models
• Deep NMF Topic Modeling
• Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference
• Beta diffusion trees and hierarchical feature allocations
• Conversational Structure Aware and Context Sensitive Topic Model for Online Discussions
• Topic Browsing for Research Papers with Hierarchical Latent Tree Analysis
• Bayesian Models for Heterogeneous Personalized Health Data
• Nested Hierarchical Dirichlet Processes for Multi-Level Non-Parametric Admixture Modeling
• Nested Dirichlet models for unsupervised attack pattern detection in honeypot data
• Gamma Belief Networks
• A Common Atom Model for the Bayesian Nonparametric Analysis of Nested Data
• Inferring Hierarchical Mixture Structures: A Bayesian Nonparametric Approach
• Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance
• Deep Neural Network for Analysis of DNA Methylation Data
• Infinite Mixture of Inverted Dirichlet Distributions
• Understanding and Enhancing the Use of Context for Machine Translation
• Modeling Hierarchical Usage Context for Software Exceptions based on Interaction Data
• A Novel Document Generation Process for Topic Detection based on Hierarchical Latent Tree Models
• Handling Collocations in Hierarchical Latent Tree Analysis for Topic Modeling
• Scalable Inference for Nested Chinese Restaurant Process Topic Models
• Blind Image Denoising via Dependent Dirichlet Process Tree