Hierarchical relational models for document networks

Abstract

We develop the relational topic model (RTM), a hierarchical model of both network structure and node attributes. We focus on document networks, where the attributes of each document are its words, that is, discrete observations taken from a fixed vocabulary. For each pair of documents, the RTM models their link as a binary random variable that is conditioned on their contents. The model can be used to summarize a network of documents, predict links between them, and predict words within them. We derive efficient inference and estimation algorithms based on variational methods that take advantage of sparsity and scale with the number of links. We evaluate the predictive performance of the RTM for large networks of scientific abstracts, web documents, and geographically tagged news.

Published in at http://dx.doi.org/10.1214/09-AOAS309 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (8)

• Supervised Topic Models
• Mixed membership stochastic blockmodels
• A Bayesian Approach to Network Modularity
• Variational inference for large-scale models of discrete choice
• Describing disability through individual-level mixture models for multivariate binary data
• Syntactic Topic Models
• Component models for large networks
• Latent Topic Models for Hypertext

Cited by in corpus (24)

• A Review of Stochastic Block Models and Extensions for Graph Clustering
• Covariate-assisted spectral clustering
• Representation Learning for Natural Language Processing
• Learning Topic Models by Belief Propagation
• Scalable Text and Link Analysis with Mixed-Topic Link Models
• Twitter-Network Topic Model: A Full Bayesian Treatment for Social Network and Text Modeling
• A Topic Modeling Toolbox Using Belief Propagation
• Bibliographic Analysis with the Citation Network Topic Model
• Joint Inference of Multiple Label Types in Large Networks
• Bibliographic Analysis on Research Publications using Authors, Categorical Labels and the Citation Network
• Computer-Assisted Text Analysis for Social Science: Topic Models and Beyond
• Community detection with nodal information
• Literature Survey on Interplay of Topics, Information Diffusion and Connections on Social Networks
• Contextual Stochastic Block Model: Sharp Thresholds and Contiguity
• Topic-adjusted visibility metric for scientific articles
• Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship
• Topic Modeling Using Distributed Word Embeddings
• A Case Study on the Impact of Similarity Measure on Information Retrieval based Software Engineering Tasks
• Improving unsupervised neural aspect extraction for online discussions using out-of-domain classification
• Discovering Political Topics in Facebook Discussion threads with Graph Contextualization
• Higher-Order Markov Tag-Topic Models for Tagged Documents and Images
• Nonparametric Relational Topic Models through Dependent Gamma Processes
• From Community Detection to Community Profiling
• Joint Modeling of Topics, Citations, and Topical Authority in Academic Corpora