The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service

Abstract

Human conversations are complicated and building a human-like dialogue agent is an extremely challenging task. With the rapid development of deep learning techniques, data-driven models become more and more prevalent which need a huge amount of real conversation data. In this paper, we construct a large-scale real scenario Chinese E-commerce conversation corpus, JDDC, with more than 1 million multi-turn dialogues, 20 million utterances, and 150 million words. The dataset reflects several characteristics of human-human conversations, e.g., goal-driven, and long-term dependency among the context. It also covers various dialogue types including task-oriented, chitchat and question-answering. Extra intent information and three well-annotated challenge sets are also provided. Then, we evaluate several retrieval-based and generative models to provide basic benchmark performance on the JDDC corpus. And we hope JDDC can serve as an effective testbed and benefit the development of fundamental research in dialogue task

arXiv commentThis paper is accepted by LREC 2020 (International Conference on Language Resources and Evaluation )

References in corpus (5)

• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• Building a Conversational Agent Overnight with Dialogue Self-Play
• Modeling Multi-turn Conversation with Deep Utterance Aggregation
• Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
• A Manually Annotated Chinese Corpus for Non-task-oriented Dialogue Systems

Cited by in corpus (4)

• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training
• KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System
• Cross Copy Network for Dialogue Generation