Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training

Abstract

Generative dialogue models currently suffer from a number of problems which standard maximum likelihood training does not address. They tend to produce generations that (i) rely too much on copying from the context, (ii) contain repetitions within utterances, (iii) overuse frequent words, and (iv) at a deeper level, contain logical flaws. In this work we show how all of these problems can be addressed by extending the recently introduced unlikelihood loss (Welleck et al., 2019) to these cases. We show that appropriate loss functions which regularize generated outputs to match human distributions are effective for the first three issues. For the last important general issue, we show applying unlikelihood to collected data of what a model should not do is effective for improving logical consistency, potentially paving the way to generative models with greater reasoning ability. We demonstrate the efficacy of our approach across several dialogue tasks.

References in corpus (8)

• The Curious Case of Neural Text Degeneration
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• Neural Text Generation with Unlikelihood Training
• Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
• Language Models as Knowledge Bases?
• ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
• HellaSwag: Can a Machine Really Finish Your Sentence?
• Latent Relation Language Models

Cited by in corpus (5)

• Knowledge-Aware Language Model Pretraining
• TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation
• Conditional Text Generation for Harmonious Human-Machine Interaction
• Language Models not just for Pre-training: Fast Online Neural Noisy Channel Modeling
• SDA: Improving Text Generation with Self Data Augmentation