Adversarial Learning for Neural Dialogue Generation

Abstract

In this paper, drawing intuition from the Turing test, we propose using adversarial training for open-domain dialogue generation: the system is trained to produce sequences that are indistinguishable from human-generated dialogue utterances. We cast the task as a reinforcement learning (RL) problem where we jointly train two systems, a generative model to produce response sequences, and a discriminator---analagous to the human evaluator in the Turing test--- to distinguish between the human-generated dialogues and the machine-generated ones. The outputs from the discriminator are then used as rewards for the generative model, pushing the system to generate dialogues that mostly resemble human dialogues. In addition to adversarial training we describe a model for adversarial {\em evaluation} that uses success in fooling an adversary as a dialogue evaluation metric, while avoiding a number of potential pitfalls. Experimental results on several metrics, including adversarial evaluation, demonstrate that the adversarially-trained system generates higher-quality responses than previous baselines.

References in corpus (9)

• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation
• An Actor-Critic Algorithm for Sequence Prediction
• A Hierarchical Neural Autoencoder for Paragraphs and Documents
• Generative Deep Neural Networks for Dialogue: A Short Review
• Adversarial Evaluation of Dialogue Models
• Aspect-augmented Adversarial Networks for Domain Adaptation
• Deep Active Learning for Dialogue Generation

Cited by in corpus (142)

• Spectral Normalization for Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• Recent Trends in Deep Learning Based Natural Language Processing
• The Curious Case of Neural Text Degeneration
• DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Self-Supervised Video Hashing with Hierarchical Binary Auto-encoder
• Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs
• Adversarial Examples for Evaluating Reading Comprehension Systems
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Synthesizing Tabular Data using Generative Adversarial Networks
• Natural Language Processing Advancements By Deep Learning: A Survey
• NetGAN: Generating Graphs via Random Walks
• GraphGAN: Graph Representation Learning with Generative Adversarial Nets
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Texygen: A Benchmarking Platform for Text Generation Models
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Assessing The Factual Accuracy of Generated Text
• Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
• Adversarial Feature Matching for Text Generation
• DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
• Boundary-Seeking Generative Adversarial Networks
• Probabilistic Forecasting of Sensory Data with Generative Adversarial Networks - ForGAN
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
• Language Generation with Recurrent Generative Adversarial Networks without Pre-training
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• T-CGAN: Conditional Generative Adversarial Network for Data Augmentation in Noisy Time Series with Irregular Sampling
• Adversarial Neural Machine Translation
• Generating Persona Consistent Dialogues by Exploiting Natural Language Inference
• Deconvolutional Paragraph Representation Learning
• Topic-Guided Variational Autoencoders for Text Generation
• Zero-Resource Knowledge-Grounded Dialogue Generation
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Unifying Human and Statistical Evaluation for Natural Language Generation
• Deep Learning Based Chatbot Models
• Recent advances in conversational NLP : Towards the standardization of Chatbot building
• Adversarially Regularized Autoencoders
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
• A Decentralized Parallel Algorithm for Training Generative Adversarial Nets
• Neural Text Generation: A Practical Guide
• A Tutorial on Deep Latent Variable Models of Natural Language
• Description Based Text Classification with Reinforcement Learning
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• Unseen Target Stance Detection with Adversarial Domain Generalization
• Low-Resource Text Classification using Domain-Adversarial Learning
• On Evaluating and Comparing Open Domain Dialog Systems
• Neural Language Generation: Formulation, Methods, and Evaluation
• Adversarial Text Generation Without Reinforcement Learning
• Paraphrase Generation with Deep Reinforcement Learning
• DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation
• An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation
• Towards Robust Neural Machine Translation
• Exploration on Generating Traditional Chinese Medicine Prescription from Symptoms with an End-to-End method
• Adversarial Network Bottleneck Features for Noise Robust Speaker Verification
• Semi-supervised Text Regression with Conditional Generative Adversarial Networks
• OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts
• Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
• A Study of Reinforcement Learning for Neural Machine Translation
• A Hierarchical Latent Structure for Variational Conversation Modeling
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• Learning to Evaluate Image Captioning
• RubyStar: A Non-Task-Oriented Mixture Model Dialog System
• Robust Multilingual Part-of-Speech Tagging via Adversarial Training
• Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
• Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generation
• Non-Autoregressive Neural Dialogue Generation
• Decoupling Strategy and Generation in Negotiation Dialogues
• Defending Against Backdoor Attacks in Natural Language Generation
• AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks
• End-to-end Adversarial Learning for Generative Conversational Agents
• Natural Language Generation with Neural Variational Models
• ACtuAL: Actor-Critic Under Adversarial Learning
• Improved Natural Language Generation via Loss Truncation
• Deep Learning Based on Generative Adversarial and Convolutional Neural Networks for Financial Time Series Predictions
• Probing Neural Dialog Models for Conversational Understanding
• Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation
• Generative Visual Dialogue System via Adaptive Reasoning and Weighted Likelihood Estimation
• Extending Neural Generative Conversational Model using External Knowledge Sources
• ColdGANs: Taming Language GANs with Cautious Sampling Strategies
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• Report from the NSF Future Directions Workshop, Toward User-Oriented Agents: Research Directions and Challenges
• Generating Responses Expressing Emotion in an Open-domain Dialogue System
• An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
• Reinforced Extractive Summarization with Question-Focused Rewards
• The Rapidly Changing Landscape of Conversational Agents
• Visual Forecasting by Imitating Dynamics in Natural Sequences
• A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
• CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation
• Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity
• Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
• Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots
• DLGNet: A Transformer-based Model for Dialogue Response Generation
• SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation
• How To Evaluate Your Dialogue System: Probe Tasks as an Alternative for Token-level Evaluation Metrics
• OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts
• Towards Solving Text-based Games by Producing Adaptive Action Spaces
• Learning from Fact-checkers: Analysis and Generation of Fact-checking Language
• Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning
• $A^{4}NT$: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation
• Affective Neural Response Generation
• Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
• Data Augmentation for Enhancing EEG-based Emotion Recognition with Deep Generative Models
• Enhancing Dialogue Generation via Multi-Level Contrastive Learning
• Product Title Refinement via Multi-Modal Generative Adversarial Learning
• Text Generation Based on Generative Adversarial Nets with Latent Variable
• Towards Amortized Ranking-Critical Training for Collaborative Filtering
• Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training
• Automatic Detection of Vague Words and Sentences in Privacy Policies
• Controllable and Diverse Text Generation in E-commerce
• Adversarial Sub-sequence for Text Generation
• Compression and Localization in Reinforcement Learning for ATARI Games
• Contextual Topic Modeling For Dialog Systems
• Adversarial Bootstrapping for Dialogue Model Training
• Adversarial Neural Networks for Cross-lingual Sequence Tagging
• An Auxiliary Classifier Generative Adversarial Framework for Relation Extraction
• Towards Automatic Generation of Entertaining Dialogues in Chinese Crosstalks
• Adversarial Learning of Task-Oriented Neural Dialog Models
• Domain Agnostic Real-Valued Specificity Prediction
• A Bi-Encoder LSTM Model For Learning Unstructured Dialogs
• An Adversarial Approach for Explainable AI in Intrusion Detection Systems
• Distributional Discrepancy: A Metric for Unconditional Text Generation
• Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning
• Posterior-GAN: Towards Informative and Coherent Response Generation with Posterior Generative Adversarial Network
• A Text Reassembling Approach to Natural Language Generation
• Transferable Natural Language Interface to Structured Queries aided by Adversarial Generation
• Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
• Generative Adversarial Nets for Multiple Text Corpora
• Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis
• Efficient text generation of user-defined topic using generative adversarial networks
• Predict and Use Latent Patterns for Short-Text Conversation
• Generation of Synthetic Electronic Medical Record Text
• The Detection of Distributional Discrepancy for Text Generation
• Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
• Spectrum and Prosody Conversion for Cross-lingual Voice Conversion with CycleGAN
• Do Encoder Representations of Generative Dialogue Models Encode Sufficient Information about the Task ?
• FinBrain: When Finance Meets AI 2.0
• Concept Formation and Dynamics of Repeated Inference in Deep Generative Models
• REAM$♯$: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generation
• Augmenting Machine Learning with Information Retrieval to Recommend Real Cloned Code Methods for Code Completion
• Face-Focused Cross-Stream Network for Deception Detection in Videos
• AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions