A network approach to topic models

Abstract

One of the main computational and scientific challenges in the modern age is to extract useful information from unstructured texts. Topic models are one popular machine-learning approach which infers the latent topical structure of a collection of documents. Despite their success --- in particular of its most widely used variant called Latent Dirichlet Allocation (LDA) --- and numerous applications in sociology, history, and linguistics, topic models are known to suffer from severe conceptual and practical problems, e.g. a lack of justification for the Bayesian priors, discrepancies with statistical properties of real texts, and the inability to properly choose the number of topics. Here we obtain a fresh view on the problem of identifying topical structures by relating it to the problem of finding communities in complex networks. This is achieved by representing text corpora as bipartite networks of documents and words. By adapting existing community-detection methods -- using a stochastic block model (SBM) with non-parametric priors -- we obtain a more versatile and principled framework for topic modeling (e.g., it automatically detects the number of topics and hierarchically clusters both the words and documents). The analysis of artificial and real corpora demonstrates that our SBM approach leads to better topic models than LDA in terms of statistical model selection. More importantly, our work shows how to formally relate methods from community detection and topic modeling, opening the possibility of cross-fertilization between these two fields.

22 pages, 10 figures, code available at https://topsbm.github.io/

References in corpus (8)

• Stochastic blockmodels and community structure in networks
• Phase transition in the detection of modules in sparse networks
• Parsimonious module inference in large networks
• Model selection and hypothesis testing for large-scale network models with overlapping groups
• The Author-Topic Model for Authors and Documents
• Consistencies and inconsistencies between model selection and link prediction in networks
• Niche as a determinant of word fate in online groups
• Dense Power-law Networks and Simplicial Complexes

Cited by in corpus (34)

• Networks beyond pairwise interactions: structure and dynamics
• A Review of Stochastic Block Models and Extensions for Graph Clustering
• Selective Exposure shapes the Facebook News Diet
• A Clarified Typology of Core-Periphery Structure in Networks
• Community Detection in Bipartite Networks with Stochastic Blockmodels
• Investigating Entropy for Extractive Document Summarization
• Scholastic: Graphical Human-Al Collaboration for Inductive and Interpretive Text Analysis
• Constructing dynamic residential energy lifestyles using Latent Dirichlet Allocation
• Detection of Community Structures in Networks with Nodal Features based on Generative Probabilistic Approach
• Sentiment and structure in word co-occurrence networks on Twitter
• Multilayer Networks for Text Analysis with Multiple Data Types
• Symptom extraction from the narratives of personal experiences with COVID-19 on Reddit
• Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing
• Where is your field going? A Machine Learning approach to study the relative motion of the domains of Physics
• A new system for evaluating brand importance: A use case from the fashion industry
• Semantic Hypergraphs
• Evolution of biomedical innovation quantified via billions of distinct article-level MeSH keyword combinations
• Discovering Significant Topics from Legal Decisions with Selective Inference
• Term-community-based topic detection with variable resolution
• Determination of the Number of Topics Intrinsically: Is It Possible?
• Empirical Analysis of Zipf's Law, Power Law, and Lognormal Distributions in Medical Discharge Reports
• How research programs come apart: the example of supersymmetry and the disunity of physics
• Is it feasible to detect FLOSS version release events from textual messages? A case study on Stack Overflow
• TeCoMiner: Topic Discovery Through Term Community Detection
• Deep NMF Topic Modeling
• Drifts and Shifts: Characterizing the Evolution of Users Interests on Reddit
• The Concept of Decentralization Through Time and Disciplines: A Quantitative Exploration
• Synthetic graphs for link prediction benchmarking
• Graph-based open-ended survey on concerns related to COVID-19
• Negative Ties Highlight Hidden Extremes in Social Media Polarization
• The Power of Communities: A Text Classification Model with Automated Labeling Process Using Network Community Detection
• TopicModel4J: A Java Package for Topic Models
• Measuring Scientific Broadness
• Co-factor analysis of citation networks