Data Distillation for Controlling Specificity in Dialogue Generation

Abstract

People speak at different levels of specificity in different situations. Depending on their knowledge, interlocutors, mood, etc.} A conversational agent should have this ability and know when to be specific and when to be general. We propose an approach that gives a neural network--based conversational agent this ability. Our approach involves alternating between \emph{data distillation} and model training : removing training examples that are closest to the responses most commonly produced by the model trained from the last round and then retrain the model on the remaining dataset. Dialogue generation models trained with different degrees of data distillation manifest different levels of specificity. We then train a reinforcement learning system for selecting among this pool of generation models, to choose the best level of specificity for a given input. Compared to the original generative model trained without distillation, the proposed system is capable of generating more interesting and higher-quality responses, in addition to appropriately adjusting specificity depending on the context. Our research constitutes a specific case of a broader approach involving training multiple subsystems from a single dataset distinguished by differences in a specific property one wishes to model. We show that from such a set of subsystems, one can use reinforcement learning to build a system that tailors its output to different input contexts at test time.

References in corpus (10)

• A Neural Conversational Model
• Sequence Level Training with Recurrent Neural Networks
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Deep Reinforcement Learning for Dialogue Generation
• Reinforcement Learning Neural Turing Machines - Revised
• Generative Deep Neural Networks for Dialogue: A Short Review
• Adversarial Evaluation of Dialogue Models
• Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
• Scaling the Indian Buffet Process via Submodular Maximization
• Deep Active Learning for Dialogue Generation

Cited by in corpus (6)

• Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators
• Generating Responses Expressing Emotion in an Open-domain Dialogue System
• Self-Attention-Based Message-Relevant Response Generation for Neural Conversation Model
• Towards Efficiently Diversifying Dialogue Generation via Embedding Augmentation
• Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization
• Automatic Evaluation of Neural Personality-based Chatbots