Continuous Time Dynamic Topic Models

Abstract

In this paper, we develop the continuous time dynamic topic model (cDTM). The cDTM is a dynamic topic model that uses Brownian motion to model the latent topics through a sequential collection of documents, where a "topic" is a pattern of word use that we expect to evolve over the course of the collection. We derive an efficient variational approximate inference algorithm that takes advantage of the sparsity of observations in text, a property that lets us easily handle many time points. In contrast to the cDTM, the original discrete-time dynamic topic model (dDTM) requires that time be discretized. Moreover, the complexity of variational inference for the dDTM grows quickly as time granularity increases, a drawback which limits fine-grained discretization. We demonstrate the cDTM on two news corpora, reporting both predictive perplexity and the novel task of time stamp prediction.

Appears in Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI2008)

References in corpus (1)

• Probabilistic Latent Semantic Analysis

Cited by in corpus (37)

• Learning under Concept Drift: an Overview
• Kernel Topic Models
• Stochastic Variational Inference
• Scalable Generalized Dynamic Topic Models
• The Dynamic Embedded Topic Model
• Mind the Gap: Assessing Temporal Generalization in Neural Language Models
• Towards Bayesian Deep Learning: A Framework and Some Existing Methods
• Dirichlet Process with Mixed Random Measures: A Nonparametric Topic Model for Labeled Data
• A Survey on Bayesian Deep Learning
• Dating Texts without Explicit Temporal Cues
• Discovering shared and individual latent structure in multiple time series
• Structured Black Box Variational Inference for Latent Time Series Models
• A unifying representation for a class of dependent random measures
• Semantic Scan: Detecting Subtle, Spatially Localized Events in Text Streams
• Item Recommendation with Continuous Experience Evolution of Users using Brownian Motion
• POMDPs in Continuous Time and Discrete Spaces
• Improving unsupervised neural aspect extraction for online discussions using out-of-domain classification
• Mixed Membership Models for Time Series
• Recurrent Point Processes for Dynamic Review Models
• Scalable Bayesian reduced-order models for high-dimensional multiscale dynamical systems
• Identifying Hidden Buyers in Darknet Markets via Dirichlet Hawkes Process
• Recurrent Point Review Models
• Local Exchangeability
• Deep Poisson gamma dynamical systems
• Local Space-Time Smoothing for Version Controlled Documents
• Transforming Graph Representations for Statistical Relational Learning
• Variational Inference in Nonconjugate Models
• Discovering topics with neural topic models built from PLSA assumptions
• Dynamic Hierarchical Dirichlet Process for Abnormal Behaviour Detection in Video
• Dynamic Topic Language Model on Heterogeneous Children's Mental Health Clinical Notes
• Jointly Dynamic Topic Model for Recognition of Lead-lag Relationship in Two Text Corpora
• Evolving Voices Based on Temporal Poisson Factorisation
• Discovering topic structures of a temporally evolving document corpus
• Visualization of Clandestine Labs from Seizure Reports: Thematic Mapping and Data Mining Research Directions
• Recurrent Coupled Topic Modeling over Sequential Documents
• Monitoring geometrical properties of word embeddings for detecting the emergence of new topics
• Analysis of Computational Science Papers from ICCS 2001-2016 using Topic Modeling and Graph Theory