Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2

Abstract

With the COVID-19 pandemic, there is a growing urgency for medical community to keep up with the accelerating growth in the new coronavirus-related literature. As a result, the COVID-19 Open Research Dataset Challenge has released a corpus of scholarly articles and is calling for machine learning approaches to help bridging the gap between the researchers and the rapidly growing publications. Here, we take advantage of the recent advances in pre-trained NLP models, BERT and OpenAI GPT-2, to solve this challenge by performing text summarization on this dataset. We evaluate the results using ROUGE scores and visual inspection. Our model provides abstractive and comprehensive information based on keywords extracted from the original articles. Our work can help the the medical community, by providing succinct summaries of articles for which the abstract are not already available.

References in corpus (2)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Leveraging BERT for Extractive Text Summarization on Lectures

Cited by in corpus (8)

• Abstractive Text Summarization: State of the Art, Challenges, and Improvements
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• CliniDigest: A Case Study in Large Language Model Based Large-Scale Summarization of Clinical Trial Descriptions
• Abstractive Text Summarization Using the BRIO Training Paradigm
• Meta-research on COVID-19: An overview of the early trends
• Ontology-based and User-focused Automatic Text Summarization (OATS): Using COVID-19 Risk Factors as an Example
• Fine-tuning GPT-3 for Russian Text Summarization
• GeSERA: General-domain Summary Evaluation by Relevance Analysis