Neural Variational Inference for Text Processing

Abstract

Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for generative and conditional models of text. While traditional variational methods derive an analytic approximation for the intractable distributions over latent variables, here we construct an inference network conditioned on the discrete text input to provide the variational distribution. We validate this framework on two very different text modelling applications, generative document modelling and supervised question answering. Our neural variational document model combines a continuous stochastic document representation with a bag-of-words generative model and achieves the lowest reported perplexities on two standard test corpora. The neural answer selection model employs a stochastic representation layer within an attention mechanism to extract the semantics between a question and answer pair. On two question answering benchmarks this model exceeds all previous published benchmarks.

ICML 2016

Cited by in corpus (16)

• An Introduction to Variational Autoencoders
• Dynamical Variational Autoencoders: A Comprehensive Review
• Classification Aware Neural Topic Model and its Application on a New COVID-19 Disinformation Corpus
• Quantum Language Model with Entanglement Embedding for Question Answering
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• A Survey of COVID-19 Misinformation: Datasets, Detection Techniques and Open Issues
• DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
• Learning document embeddings along with their uncertainties
• vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
• Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums
• Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning
• SenGen: Sentence Generating Neural Variational Topic Model
• Do sequence-to-sequence VAEs learn global features of sentences?
• Recurrent Point Review Models
• Dynamic Topic Language Model on Heterogeneous Children's Mental Health Clinical Notes
• Learning Sparsity of Representations with Discrete Latent Variables