DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

Abstract

We present a large, tunable neural conversational response generation model, DialoGPT (dialogue generative pre-trained transformer). Trained on 147M conversation-like exchanges extracted from Reddit comment chains over a period spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch transformer to attain a performance close to human both in terms of automatic and human evaluation in single-turn dialogue settings. We show that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems. The pre-trained model and training pipeline are publicly released to facilitate research into neural response generation and the development of more intelligent open-domain dialogue systems.

arXiv commentAccepted by ACL 2020 system demonstration

References in corpus (3)

• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
• DLGNet: A Transformer-based Model for Dialogue Response Generation

Cited by in corpus (58)

• A Simple Language Model for Task-Oriented Dialogue
• Recipes for Safety in Open-domain Chatbots
• Large Language Models Can Be Strong Differentially Private Learners
• Zero-Resource Knowledge-Grounded Dialogue Generation
• Challenges in Building Intelligent Open-domain Dialog Systems
• Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
• Automated Quality Assessment of Cognitive Behavioral Therapy Sessions Through Highly Contextualized Language Representations
• Controlling Style in Generated Dialogue
• Few-shot Natural Language Generation for Task-Oriented Dialog
• STAR: A Schema-Guided Dialog Dataset for Transfer Learning
• BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
• Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
• An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation
• Gunrock 2.0: A User Adaptive Social Conversational System
• Profile Consistency Identification for Open-domain Dialogue Agents
• OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts
• CL4AC: A Contrastive Loss for Audio Captioning
• Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
• Robust Conversational AI with Grounded Text Generation
• Response Generation with Context-Aware Prompt Learning
• MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
• Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generation
• Code-switched inspired losses for generic spoken dialog representations
• Non-Autoregressive Neural Dialogue Generation
• On the Generation of Medical Dialogues for COVID-19
• Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?
• A Tailored Pre-Training Model for Task-Oriented Dialog Generation
• TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue System
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
• ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
• Multi-Domain Dialogue Acts and Response Co-Generation
• Dialogue Response Ranking Training with Large-Scale Human Feedback Data
• GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
• Controllable Abstractive Dialogue Summarization with Sketch Supervision
• Conditional Text Generation for Harmonious Human-Machine Interaction
• Knowledge-Grounded Dialogue Generation with Pre-trained Language Models
• A Large-Scale Chinese Short-Text Conversation Dataset
• OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts
• Learning to Retrieve Entity-Aware Knowledge and Generate Responses with Copy Mechanism for Task-Oriented Dialogue Systems
• Assessing Political Prudence of Open-domain Chatbots
• Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
• NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment
• MixingBoard: a Knowledgeable Stylized Integrated Text Generation Platform
• Cue-word Driven Neural Response Generation with a Shrinking Vocabulary
• A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19
• Open-Domain Dialogue Generation Based on Pre-trained Language Models
• TED-net: Convolution-free T2T Vision Transformer-based Encoder-decoder Dilation network for Low-dose CT Denoising
• Pretrained Language Models for Dialogue Generation with Multiple Input Sources
• MEEP: An Open-Source Platform for Human-Human Dialog Collection and End-to-End Agent Training
• Multi-turn Response Selection using Dialogue Dependency Relations
• A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation
• Probing Task-Oriented Dialogue Representation from Language Models
• Local Knowledge Powered Conversational Agents
• Which Kind Is Better in Open-domain Multi-turn Dialog,Hierarchical or Non-hierarchical Models? An Empirical Study
• Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging
• StyleDGPT: Stylized Response Generation with Pre-trained Language Models
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention
• Towards meaningful, grounded conversations with intelligent agents