DeepTitle -- Leveraging BERT to generate Search Engine Optimized Headlines
arXiv:2107.10935
Abstract
Automated headline generation for online news articles is not a trivial task - machine generated titles need to be grammatically correct, informative, capture attention and generate search traffic without being "click baits" or "fake news". In this paper we showcase how a pre-trained language model can be leveraged to create an abstractive news headline generator for German language. We incorporate state of the art fine-tuning techniques for abstractive text summarization, i.e. we use different optimizers for the encoder and decoder where the former is pre-trained and the latter is trained from scratch. We modify the headline generation to incorporate frequently sought keywords relevant for search engine optimization. We conduct experiments on a German news data set and achieve a ROUGE-L-gram F-score of 40.02. Furthermore, we address the limitations of ROUGE for measuring the quality of text summarization by introducing a sentence similarity metric and human evaluation.
arXiv comment9 pages, 4 figures
References in corpus (5)
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- Text Summarization with Pretrained Encoders
- Generating News Headlines with Recurrent Neural Networks
- ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training