Pchatbot: A Large-Scale Dataset for Personalized Chatbot
arXiv:2009.13284
Abstract
Natural language dialogue systems raise great attention recently. As many dialogue models are data-driven, high-quality datasets are essential to these systems. In this paper, we introduce Pchatbot, a large-scale dialogue dataset that contains two subsets collected from Weibo and Judicial forums respectively. To adapt the raw dataset to dialogue systems, we elaborately normalize the raw dataset via processes such as anonymization, deduplication, segmentation, and filtering. The scale of Pchatbot is significantly larger than existing Chinese datasets, which might benefit the data-driven models. Besides, current dialogue datasets for personalized chatbot usually contain several persona sentences or attributes. Different from existing datasets, Pchatbot provides anonymized user IDs and timestamps for both posts and responses. This enables the development of personalized dialogue models that directly learn implicit user personality from the user's dialogue history. Our preliminary experimental study benchmarks several state-of-the-art dialogue models to provide a comparison for future work. The dataset can be publicly accessed at Github.
arXiv commentCamera-ready version, SIGIR 2021 (Resource Track), the dataset and codes are available at https://github.com/qhjqhj00/Pchatbot
References in corpus (7)
- Sequence to Sequence Learning with Neural Networks
- A Neural Conversational Model
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
- Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
- Topic Aware Neural Response Generation
- A Persona-Based Neural Conversation Model
- Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems