SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline

Abstract

The Scholarly Document Processing (SDP) workshop is to encourage more efforts on natural language understanding of scientific task. It contains three shared tasks and we participate in the LongSumm shared task. In this paper, we describe our text summarization system, SciSummPip, inspired by SummPip (Zhao et al., 2020) that is an unsupervised text summarization system for multi-document in news domain. Our SciSummPip includes a transformer-based language model SciBERT (Beltagy et al., 2019) for contextual sentence representation, content selection with PageRank (Page et al., 1999), sentence graph construction with both deep and linguistic information, sentence graph clustering and within-graph summary generation. Our work differs from previous method in that content selection and a summary length constraint is applied to adapt to the scientific domain. The experiment results on both training dataset and blind test dataset show the effectiveness of our method, and we empirically verify the robustness of modules used in SciSummPip with BERTScore (Zhang et al., 2019a).

References in corpus (17)

• Sequence to Sequence Learning with Neural Networks
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• BERTScore: Evaluating Text Generation with BERT
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• Text Summarization with Pretrained Encoders
• Neural Summarization by Extracting Sentences and Words
• Variations of the Similarity Function of TextRank for Automated Summarization
• Get To The Point: Summarization with Pointer-Generator Networks
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Sample Efficient Text Summarization Using a Single Pre-Trained Transformer
• HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
• SciBERT: A Pretrained Language Model for Scientific Text
• Heterogeneous Graph Neural Networks for Extractive Document Summarization
• Neural Extractive Text Summarization with Syntactic Compression
• An Entity-Driven Framework for Abstractive Summarization
• Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning
• Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization