Kernel Topic Models

Abstract

Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with elements of a Hilbert space, admitting kernel topic models (KTM), modelling temporal, spatial, hierarchical, social and other structure between documents. The main challenge is efficient approximate inference on the latent Gaussian. We present an approximate algorithm cast around a Laplace approximation in a transformed basis. The KTM can also be interpreted as a type of Gaussian process latent variable model, or as a topic model conditional on document features, uncovering links between earlier work in these areas.

References in corpus (3)

• Continuous Time Dynamic Topic Models
• Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression
• Elliptical slice sampling

Cited by in corpus (9)

• Factorized Multi-Modal Topic Model
• Laplace Redux -- Effortless Bayesian Deep Learning
• Scalable Generalized Dynamic Topic Models
• Dirichlet Graph Variational Autoencoder
• Fast Predictive Uncertainty for Classification with Bayesian Deep Networks
• Unsupervised Machine Commenting with Neural Variational Topic Model
• Hierarchically Clustered Representation Learning
• Variational Gaussian Topic Model with Invertible Neural Projections
• Graph-Driven Generative Models for Heterogeneous Multi-Task Learning