Semantic-Enhanced Explainable Finetuning for Open-Domain Dialogues
arXiv:2106.03065
Abstract
This paper propose to combine pretrained language models with the modular dialogue paradigm for open-domain dialogue modeling. Our method, semantic-enhanced finetuning, instantiates conversation understanding, planning, and response generation as a language model finetuning task. At inference, we disentangle semantic and token variations by specifying sampling methods and constraints for each module separately. For training and evaluation, we present X-Weibo, a Chinese multi-turn open-domain dialogue dataset with automatic annotation for emotions, DAs, and topical words. Experiments show that semantic-enhanced finetuning outperforms strong baselines on non-semantic and semantic metrics, improves the human-evaluated relevance, coherence, and informativeness, and exhibits considerable controllability over semantic variables.
arXiv commentUnder review
References in corpus (9)
- Sequence to Sequence Learning with Neural Networks
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
- Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
- Hierarchical Neural Story Generation
- CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model
- Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
- COMET: Commonsense Transformers for Automatic Knowledge Graph Construction
- Policy-Driven Neural Response Generation for Knowledge-Grounded Dialogue Systems
- Multi-task learning for Joint Language Understanding and Dialogue State Tracking