Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation
arXiv:2003.01680
Abstract
Domain adaptation has recently become a key problem in dialogue systems research. Deep learning, while being the preferred technique for modeling such systems, works best given massive training data. However, in the real-world scenario, such resources aren't available for every new domain, so the ability to train with a few dialogue examples can be considered essential. Pre-training on large data sources and adapting to the target data has become the standard method for few-shot problems within the deep learning framework. In this paper, we present the winning entry at the fast domain adaptation task of DSTC8, a hybrid generative-retrieval model based on GPT-2 fine-tuned to the multi-domain MetaLWOz dataset. Robust and diverse in response generation, our model uses retrieval logic as a fallback, being SoTA on MetaLWOz in human evaluation (>4% improvement over the 2nd place system) and attaining competitive generalization performance in adaptation to the unseen MultiWOZ dataset.
arXiv commentPresented at DSTC8@AAAI 2020
References in corpus (10)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- A Neural Conversational Model
- TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
- A Diversity-Promoting Objective Function for Neural Conversation Models
- Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation
- ConvLab: Multi-Domain End-to-End Dialog System Platform
- Few-Shot Generalization Across Dialogue Tasks
- Just Sort It! A Simple and Effective Approach to Active Preference Learning
- Zero-Shot Dialog Generation with Cross-Domain Latent Actions
- Neural Net Models for Open-Domain Discourse Coherence