MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
arXiv:2009.12005
Abstract
In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to "carryover" the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20\% training data, and 3) Lev greatly improves the inference efficiency.
EMNLP 2020 camera ready
References in corpus (18)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
- Towards a Human-like Open-Domain Chatbot
- A Simple Language Model for Task-Oriented Dialogue
- DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
- SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching
- Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering
- Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models
- Non-Autoregressive Dialog State Tracking
- Attention over Parameters for Dialogue Systems
- Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context
- Teacher-Student Framework Enhanced Multi-domain Dialogue Generation
- Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
- Toward Scalable Neural Dialogue State Tracking Model
- Efficient Dialogue State Tracking by Selectively Overwriting Memory
- Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning