Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness
arXiv:2004.14008
Abstract
Large-scale dialogue datasets have recently become available for training neural dialogue agents. However, these datasets have been reported to contain a non-negligible number of unacceptable utterance pairs. In this paper, we propose a method for scoring the quality of utterance pairs in terms of their connectivity and relatedness. The proposed scoring method is designed based on findings widely shared in the dialogue and linguistics research communities. We demonstrate that it has a relatively good correlation with the human judgment of dialogue quality. Furthermore, the method is applied to filter out potentially unacceptable utterance pairs from a large-scale noisy dialogue corpus to ensure its quality. We experimentally confirm that training data filtered by the proposed method improves the quality of neural dialogue agents in response generation.
arXiv comment18 pages, Accepted at The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)
References in corpus (7)
- Sequence to Sequence Learning with Neural Networks
- A Neural Conversational Model
- Wizard of Wikipedia: Knowledge-Powered Conversational agents
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
- Topic Aware Neural Response Generation
- RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems
- deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets