SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization
arXiv:1911.12237 · doi:10.18653/v1/D19-5409
Abstract
This paper introduces the SAMSum Corpus, a new dataset with abstractive dialogue summaries. We investigate the challenges it poses for automated summarization by testing several models and comparing their results with those obtained on a corpus of news articles. We show that model-generated summaries of dialogues achieve higher ROUGE scores than the model-generated summaries of news -- in contrast with human evaluators' judgement. This suggests that a challenging task of abstractive dialogue summarization requires dedicated models and non-standard quality measures. To our knowledge, our study is the first attempt to introduce a high-quality chat-dialogues corpus, manually annotated with abstractive summarizations, which can be used by the research community for further studies.
Attachment contains the described dataset archived in 7z format. Please see the attached readme and licence. Update of the previous version: changed formats of train/val/test files in corpus.7z
References in corpus (3)
Cited by in corpus (35)
- Multitask Prompted Training Enables Zero-Shot Task Generalization
- Hierarchical Learning for Generation with Long Source Sequences
- Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
- Digital Forgetting in Large Language Models: A Survey of Unlearning Methods
- An Analysis of Abstractive Text Summarization Using Pre-trained Models
- GO FIGURE: A Meta Evaluation of Factuality in Summarization
- Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents
- Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly
- DialogSum: A Real-Life Scenario Dialogue Summarization Dataset
- AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
- FLEdge: Benchmarking Federated Machine Learning Applications in Edge Computing Systems
- Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization
- Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs
- GupShup: An Annotated Corpus for Abstractive Summarization of Open-Domain Code-Switched Conversations
- Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization
- Controllable Abstractive Dialogue Summarization with Sketch Supervision
- From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information
- ORCHID: A Chinese Debate Corpus for Target-Independent Stance Detection and Argumentative Dialogue Summarization
- A Hybrid Strategy for Chat Transcript Summarization
- Enhancing Semantic Understanding with Self-supervised Methods for Abstractive Dialogue Summarization
- MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
- CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarization
- Dynamic Sliding Window for Meeting Summarization
- Large Language Models in Mental Health: A Systematic Review of Applications, Innovations, and Ethical Challenges
- CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP
- EmailSum: Abstractive Email Thread Summarization
- An Exploratory Study on Long Dialogue Summarization: What Works and What's Next
- SummerTime: Text Summarization Toolkit for Non-experts
- Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization
- TWEETSUMM -- A Dialog Summarization Dataset for Customer Service
- Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining
- Controllable Neural Dialogue Summarization with Personal Named Entity Planning
- Generative Fuzzy System for Sequence Generation
- Hierarchical Summarization for Longform Spoken Dialog
- Does Pretraining for Summarization Require Knowledge Transfer?