DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder

Abstract

Variational autoencoders~(VAEs) have shown a promise in data-driven conversation modeling. However, most VAE conversation models match the approximate posterior distribution over the latent variables to a simple prior such as standard normal distribution, thereby restricting the generated responses to a relatively simple (e.g., unimodal) scope. In this paper, we propose DialogWAE, a conditional Wasserstein autoencoder~(WAE) specially designed for dialogue modeling. Unlike VAEs that impose a simple distribution over the latent variables, DialogWAE models the distribution of data by training a GAN within the latent variable space. Specifically, our model samples from the prior and posterior distributions over the latent variables by transforming context-dependent random noise using neural networks and minimizes the Wasserstein distance between the two distributions. We further develop a Gaussian mixture prior network to enrich the latent space. Experiments on two popular datasets show that DialogWAE outperforms the state-of-the-art approaches in generating more coherent, informative and diverse responses.

Published as a conference paper at ICLR 2019

References in corpus (15)

• Auto-Encoding Variational Bayes
• Sequence to Sequence Learning with Neural Networks
• On the difficulty of training Recurrent Neural Networks
• Style Transfer from Non-Parallel Text by Cross-Alignment
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Topic Aware Neural Response Generation
• Adversarial Learning for Neural Dialogue Generation
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Wasserstein Auto-Encoders
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
• NSML: Meet the MLaaS platform with a real-world case study
• NSML: A Machine Learning Platform That Enables You to Focus on Your Models
• Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
• Improving Variational Encoder-Decoders in Dialogue Generation
• A Hierarchical Latent Structure for Variational Conversation Modeling

Cited by in corpus (17)

• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• Conditional Flow Variational Autoencoders for Structured Sequence Prediction
• A Tutorial on Deep Latent Variable Models of Natural Language
• An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation
• Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
• Secure Deep Graph Generation with Link Differential Privacy
• Jointly Optimizing Diversity and Relevance in Neural Response Generation
• Mixture Content Selection for Diverse Sequence Generation
• Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner
• On the Generation of Medical Question-Answer Pairs
• Towards Coherent and Cohesive Long-form Text Generation
• Improving Text Generation with Student-Forcing Optimal Transport
• A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation
• Proposal Towards a Personalized Knowledge-powered Self-play Based Ensemble Dialog System
• Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior
• Focus-Constrained Attention Mechanism for CVAE-based Response Generation
• Learning from Easy to Complex: Adaptive Multi-curricula Learning for Neural Dialogue Generation