Discovering Discrete Latent Topics with Neural Variational Inference
arXiv:1706.00359
Abstract
Topic models have been widely explored as probabilistic generative models of documents. Traditional inference methods have sought closed-form derivations for updating the models, however as the expressiveness of these models grows, so does the difficulty of performing fast and accurate inference over their parameters. This paper presents alternative neural approaches to topic modelling by providing parameterisable distributions over topics which permit training by backpropagation in the framework of neural variational inference. In addition, with the help of a stick-breaking construction, we propose a recurrent network that is able to discover a notionally unbounded number of topics, analogous to Bayesian non-parametric topic models. Experimental results on the MXM Song Lyrics, 20NewsGroups and Reuters News datasets demonstrate the effectiveness and efficiency of these neural topic models.
ICML 2017
References in corpus (3)
Cited by in corpus (27)
- A Survey of Knowledge-Enhanced Text Generation
- Classification Aware Neural Topic Model and its Application on a New COVID-19 Disinformation Corpus
- Topic-Guided Variational Autoencoders for Text Generation
- A Tutorial on Deep Latent Variable Models of Natural Language
- DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
- Dirichlet belief networks for topic structure learning
- Topic Modeling with Contextualized Word Representation Clusters
- Unsupervised Dialog Structure Learning
- Topic Memory Networks for Short Text Classification
- Learning document embeddings along with their uncertainties
- vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
- Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex
- Adaptive Parameterization for Neural Dialogue Generation
- Topic-Guided Abstractive Text Summarization: a Joint Learning Approach
- Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks
- Topic Modelling Meets Deep Neural Networks: A Survey
- Coherence-Aware Neural Topic Modeling
- Neural Topic Model via Optimal Transport
- Latent Topic Conversational Models
- TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts
- NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing
- Graph-Driven Generative Models for Heterogeneous Multi-Task Learning
- Out-of-Town Recommendation with Travel Intention Modeling
- A Comprehensive Attempt to Research Statement Generation
- Sequence-to-Sequence Learning with Latent Neural Grammars
- Enriching and Controlling Global Semantics for Text Summarization
- Have you tried Neural Topic Models? Comparative Analysis of Neural and Non-Neural Topic Models with Application to COVID-19 Twitter Data