Latent Topic Conversational Models
arXiv:1809.07070
Abstract
Latent variable models have been a preferred choice in conversational modeling compared to sequence-to-sequence (seq2seq) models which tend to generate generic and repetitive responses. Despite so, training latent variable models remains to be difficult. In this paper, we propose Latent Topic Conversational Model (LTCM) which augments seq2seq with a neural latent topic component to better guide response generation and make training easier. The neural topic component encodes information from the source sentence to build a global "topic" distribution over words, which is then consulted by the seq2seq model at each generation step. We study in details how the latent representation is learnt in both the vanilla model and LTCM. Our extensive experiments contribute to better understanding and training of conditional latent models for languages. Our results show that by sampling from the learnt latent representations, LTCM can generate diverse and interesting responses. In a subjective human evaluation, the judges also confirm that LTCM is the overall preferred option.
References in corpus (12)
- Sequence to Sequence Learning with Neural Networks
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
- Layer Normalization
- Discovering Discrete Latent Topics with Neural Variational Inference
- Neural Responding Machine for Short-Text Conversation
- Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
- TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
- Latent Variable Dialogue Models and their Diversity
- Latent Intention Dialogue Models
- Referenceless Quality Estimation for Natural Language Generation
- Steering Output Style and Topic in Neural Response Generation