Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging
arXiv:2105.10267
Abstract
In a dialogue system pipeline, a natural language generation (NLG) unit converts the dialogue direction and content to a corresponding natural language realization. A recent trend for dialogue systems is to first pre-train on large datasets and then fine-tune in a supervised manner using datasets annotated with application-specific features. Though novel behaviours can be learned from custom annotation, the required effort severely bounds the quantity of the training set, and the application-specific nature limits the reuse. In light of the recent success of data-driven approaches, we propose the novel future bridging NLG (FBNLG) concept for dialogue systems and simulators. The critical step is for an FBNLG to accept a future user or system utterance to bridge the present context towards. Future bridging enables self supervised training over annotation-free datasets, decoupled the training of NLG from the rest of the system. An FBNLG, pre-trained with massive datasets, is expected to apply in classical or new dialogue scenarios with minimal adaptation effort. We evaluate a prototype FBNLG to show that future bridging can be a viable approach to a universal few-shot NLG for task-oriented and chit-chat dialogues.
11 pages, 1 figure
References in corpus (12)
- Language Models are Few-Shot Learners
- DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
- Towards a Human-like Open-Domain Chatbot
- The Evolved Transformer
- An Information Retrieval Approach to Short Text Conversation
- Building a Conversational Agent Overnight with Dialogue Self-Play
- Training Neural Response Selection for Task-Oriented Dialogue Systems
- Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems
- Towards Topic-Guided Conversational Recommender System
- XL-Editor: Post-editing Sentences with XLNet