Leveraging BERT for Extractive Text Summarization on Lectures

Abstract

In the last two decades, automatic extractive text summarization on lectures has demonstrated to be a useful tool for collecting key phrases and sentences that best represent the content. However, many current approaches utilize dated approaches, producing sub-par outputs or requiring several hours of manual tuning to produce meaningful results. Recently, new machine learning architectures have provided mechanisms for extractive summarization through the clustering of output embeddings from deep learning models. This paper reports on the project called Lecture Summarization Service, a python based RESTful service that utilizes the BERT model for text embeddings and KMeans clustering to identify sentences closes to the centroid for summary selection. The purpose of the service was to provide students a utility that could summarize lecture content, based on their desired number of sentences. On top of the summary work, the service also includes lecture and summary management, storing content on the cloud which can be used for collaboration. While the results of utilizing BERT for extractive summarization were promising, there were still areas where the model struggled, providing feature research opportunities for further improvement.

7 Pages, First Version

References in corpus (1)

• An efficient framework for learning sentence representations

Cited by in corpus (19)

• A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis
• Big Bird: Transformers for Longer Sequences
• Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
• DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media
• Retrieving Supporting Evidence for Generative Question Answering
• Text Clustering with Large Language Model Embeddings
• MathBERT: A Pre-Trained Model for Mathematical Formula Understanding
• pyBibX -- A Python Library for Bibliometric and Scientometric Analysis Powered with Artificial Intelligence Tools
• Attention-based Clinical Note Summarization
• MultiGBS: A multi-layer graph approach to biomedical summarization
• CAiRE-COVID: A Question Answering and Query-focused Multi-Document Summarization System for COVID-19 Scholarly Information Management
• VendorLink: An NLP approach for Identifying & Linking Vendor Migrants & Potential Aliases on Darknet Markets
• Fine-tuning Strategies for Domain Specific Question Answering under Low Annotation Budget Constraints
• Connecting the Dots Between Fact Verification and Fake News Detection
• NLPBK at VLSP-2020 shared task: Compose transformer pretrained models for Reliable Intelligence Identification on Social network
• Unsupervised paradigm for information extraction from transcripts using BERT
• AI based Presentation Creator With Customized Audio Content Delivery
• Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories
• Weakly-Supervised Methods for Suicide Risk Assessment: Role of Related Domains