An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics

Abstract

Long documents such as academic articles and business reports have been the standard format to detail out important issues and complicated subjects that require extra attention. An automatic summarization system that can effectively condense long documents into short and concise texts to encapsulate the most important information would thus be significant in aiding the reader's comprehension. Recently, with the advent of neural architectures, significant research efforts have been made to advance automatic text summarization systems, and numerous studies on the challenges of extending these systems to the long document domain have emerged. In this survey, we provide a comprehensive overview of the research on long document summarization and a systematic evaluation across the three principal components of its research setting: benchmark datasets, summarization models, and evaluation metrics. For each component, we organize the literature within the context of long document summarization and conduct an empirical analysis to broaden the perspective on current research progress. The empirical analysis includes a study on the intrinsic characteristics of benchmark datasets, a multi-dimensional analysis of summarization models, and a review of the summarization evaluation metrics. Based on the overall findings, we conclude by proposing possible directions for future exploration in this rapidly growing field.

arXiv commentAccepted for publication by ACM Computing Surveys

References in corpus (17)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• BERTScore: Evaluating Text Generation with BERT
• Longformer: The Long-Document Transformer
• A Deep Reinforced Model for Abstractive Summarization
• PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
• Long Range Arena: A Benchmark for Efficient Transformers
• WikiHow: A Large Scale Text Summarization Dataset
• Assessing The Factual Accuracy of Generated Text
• Machine Learning of Generic and User-Focused Summarization
• ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks
• FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization
• Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
• CTRLsum: Towards Generic Controllable Text Summarization
• Hierarchical Learning for Generation with Long Source Sequences
• SummPip: Unsupervised Multi-Document Summarization with Sentence Graph Compression
• Can We Automate Scientific Reviewing?

Cited by in corpus (6)

• Benchmarking large language models for biomedical natural language processing applications and recommendations
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Neural Natural Language Processing for Long Texts: A Survey on Classification and Summarization
• Translating Legalese: Enhancing Public Understanding of Court Opinions with Legal Summarizers
• Markov-Enhanced Clustering for Long Document Summarization: Tackling the 'Lost in the Middle' Challenge with Large Language Models
• SurveySum: A Dataset for Summarizing Multiple Scientific Articles into a Survey Section