Few-shot Natural Language Generation for Task-Oriented Dialog

Abstract

As a crucial component in task-oriented dialog systems, the Natural Language Generation (NLG) module converts a dialog act represented in a semantic form into a response in natural language. The success of traditional template-based or statistical models typically relies on heavily annotated data, which is infeasible for new domains. Therefore, it is pivotal for an NLG system to generalize well with limited labelled data in real applications. To this end, we present FewShotWoz, the first NLG benchmark to simulate the few-shot learning setting in task-oriented dialog systems. Further, we develop the SC-GPT model. It is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine-tuned with only a few domain-specific labels to adapt to new domains. Experiments on FewShotWoz and the large Multi-Domain-WOZ datasets show that the proposed SC-GPT significantly outperforms existing methods, measured by various automatic metrics and human evaluations.

Project website: https://aka.ms/scgpt ; Code and data: https://github.com/pengbaolin/SC-GPT

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• Towards a Human-like Open-Domain Chatbot
• A Network-based End-to-End Trainable Task-oriented Dialogue System
• Plug and Play Language Models: A Simple Approach to Controlled Text Generation
• DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
• Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking
• Multi-domain Neural Network Language Generation for Spoken Dialogue Systems

Cited by in corpus (10)

• Evaluation of Text Generation: A Survey
• SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching
• Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training
• Speech-language Pre-training for End-to-end Spoken Language Understanding
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
• Neural Data-to-Text Generation with LM-based Text Augmentation
• Integrating Pretrained Language Model for Dialogue Policy Learning
• Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care
• Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling