Exploring Domain Shift in Extractive Text Summarization

Abstract

Although domain shift has been well explored in many NLP applications, it still has received little attention in the domain of extractive text summarization. As a result, the model is under-utilizing the nature of the training data due to ignoring the difference in the distribution of training sets and shows poor generalization on the unseen domain. With the above limitation in mind, in this paper, we first extend the conventional definition of the domain from categories into data sources for the text summarization task. Then we re-purpose a multi-domain summarization dataset and verify how the gap between different domains influences the performance of neural summarization models. Furthermore, we investigate four learning strategies and examine their abilities to deal with the domain shift problem. Experimental results on three different settings show their different characteristics in our new testbed. Our source code including \textit{BERT-based}, \textit{meta-learning} methods for multi-domain summarization learning and the re-purposed dataset \textsc{Multi-SUM} will be available on our project: \url{http://pfliu.com/TransferSum/}.

References in corpus (11)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Distributed Representations of Sentences and Documents
• Teaching Machines to Read and Comprehend
• A Deep Reinforced Model for Abstractive Summarization
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
• Domain Separation Networks
• Learning to Generalize: Meta-Learning for Domain Generalization
• Improving Multi-Document Summarization via Text Classification
• A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization
• Meta-Learning Multi-task Communication
• A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization

Cited by in corpus (9)

• Heterogeneous Graph Neural Networks for Extractive Document Summarization
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
• Extractive Summarization as Text Matching
• AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
• A Closer Look at Data Bias in Neural Extractive Summarization Models
• CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
• BioCopy: A Plug-And-Play Span Copy Mechanism in Seq2Seq Models
• Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining
• Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning