TNT-KID: Transformer-based Neural Tagger for Keyword Identification

Abstract

With growing amounts of available textual data, development of algorithms capable of automatic analysis, categorization and summarization of these data has become a necessity. In this research we present a novel algorithm for keyword identification, i.e., an extraction of one or multi-word phrases representing key aspects of a given document, called Transformer-based Neural Tagger for Keyword IDentification (TNT-KID). By adapting the transformer architecture for a specific task at hand and leveraging language model pretraining on a domain specific corpus, the model is capable of overcoming deficiencies of both supervised and unsupervised state-of-the-art approaches to keyword extraction by offering competitive and robust performance on a variety of different datasets while requiring only a fraction of manually labeled data required by the best performing systems. This study also offers thorough error analysis with valuable insights into the inner workings of the model and an ablation study measuring the influence of specific components of the keyword identification workflow on the overall performance.

Accepted to Natural Language Engineering journal

References in corpus (5)

• ERNIE: Enhanced Representation through Knowledge Integration
• RaKUn: Rank-based Keyword extraction via Unsupervised learning and Meta vertex aggregation
• Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings
• Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequence
• Keyphrase Generation: A Multi-Aspect Survey

Cited by in corpus (4)

• Phraseformer: Multimodal Key-phrase Extraction using Transformer and Graph Embedding
• ELSKE: Efficient Large-Scale Keyphrase Extraction
• Improving Performance of Automatic Keyword Extraction (AKE) Methods Using PoS-Tagging and Enhanced Semantic-Awareness
• Extending Neural Keyword Extraction with TF-IDF tagset matching