PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
arXiv:2006.16779
Abstract
To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.
Findings of ACL 2021. First four authors contributed equally to this work
References in corpus (8)
- Language Models are Few-Shot Learners
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Wizard of Wikipedia: Knowledge-Powered Conversational agents
- Towards a Human-like Open-Domain Chatbot
- Which Tasks Should Be Learned Together in Multi-task Learning?
- Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue
- ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
- A Unified Pre-training Framework for Conversational AI
Cited by in corpus (10)
- EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
- Pretrained Language Models for Text Generation: A Survey
- A Unified Pre-training Framework for Conversational AI
- Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances
- Learning to Retrieve Entity-Aware Knowledge and Generate Responses with Copy Mechanism for Task-Oriented Dialogue Systems
- Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking
- A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation
- Open-Domain Dialogue Generation Based on Pre-trained Language Models
- Addressing Inquiries about History: An Efficient and Practical Framework for Evaluating Open-domain Chatbot Consistency
- A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation