vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
arXiv:2307.01226 · doi:10.18653/v1/2023.findings-acl.271
Abstract
Recently, Neural Topic Models (NTM), inspired by variational autoencoders, have attracted a lot of research interest; however, these methods have limited applications in the real world due to the challenge of incorporating human knowledge. This work presents a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport. When a few keywords per topic are provided, vONTSS in the semi-supervised setting generates potential topics and optimizes topic-keyword quality and topic classification. Experiments show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity. vONTSS also supports unsupervised topic modeling. Quantitative and qualitative experiments show that vONTSS in the unsupervised setting outperforms recent NTMs on multiple aspects: vONTSS discovers highly clustered and coherent topics on benchmark datasets. It is also much faster than the state-of-the-art weakly supervised text classification method while achieving similar classification performance. We further prove the equivalence of optimal transport loss and cross-entropy loss at the global minimum.
arXiv comment24 pages, 12 figures, ACL findings 2023
References in corpus (14)
- Efficient Estimation of Word Representations in Vector Space
- Categorical Reparameterization with Gumbel-Softmax
- Character-level Convolutional Networks for Text Classification
- Semi-Supervised Learning with Deep Generative Models
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure
- Neural Variational Inference for Text Processing
- Discovering Discrete Latent Topics with Neural Variational Inference
- Weakly-Supervised Neural Text Classification
- WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling
- Discriminative Topic Mining via Category-Name Guided Text Embedding
- Keyword Assisted Embedded Topic Model
- On Low-Dimensional Projections of High-Dimensional Distributions
- A Survey on Optimal Transport for Machine Learning: Theory and Applications
- A Joint Learning Approach for Semi-supervised Neural Topic Modeling