UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2

Abstract

This paper presents our task-oriented dialog system UBAR which models task-oriented dialogs on a dialog session level. Specifically, UBAR is acquired by fine-tuning the large pre-trained unidirectional language model GPT-2 on the sequence of the entire dialog session which is composed of user utterance, belief state, database result, system act, and system response of every dialog turn. Additionally, UBAR is evaluated in a more realistic setting, where its dialog context has access to user utterances and all content it generated such as belief states, system acts, and system responses. Experimental results on the MultiWOZ datasets show that UBAR achieves state-of-the-art performances in multiple settings, improving the combined score of response generation, policy optimization, and end-to-end modeling by 4.7, 3.5, and 9.4 points respectively. Thorough analyses demonstrate that the session-level training sequence formulation and the generated dialog context are essential for UBAR to operate as a fully end-to-end task-oriented dialog system in real life. We also examine the transfer ability of UBAR to new domains with limited data and provide visualization and a case study to illustrate the advantages of UBAR in modeling on a dialog session level.

arXiv commentAccepted by AAAI 2021

References in corpus (7)

• A Simple Language Model for Task-Oriented Dialogue
• SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching
• Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering
• Global-to-local Memory Pointer Networks for Task-Oriented Dialogue
• Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models
• Recent Advances and Challenges in Task-oriented Dialog System
• MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines

Cited by in corpus (4)

• Task-Oriented Dialogue System as Natural Language Generation
• A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slots
• DORA: Toward Policy Optimization for Task-oriented Dialogue System with Efficient Context
• Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems