Label Unification for Cross-Dataset Generalization in Cybersecurity NER

Abstract

The field of cybersecurity NER lacks standard-ized labels, making it challenging to combine datasets. We investigate label unification across four cybersecurity datasets to increase data resource usability. We perform a coarse-grained label unification and conduct pairwise cross-dataset evaluations using RoBERTa models. Qualitative analysis of predictions reveals errors, limitations, and dataset differences. To address unification limitations, we propose alternative architectures including a multi-head model and a graph-based transfer model. Under our selected mapping and training setup, unified training underperformed the corresponding single-dataset baselines on all four development sets. The multi-head model with weight sharing provides no consistent benefits over single-dataset baselines, and the graph-based transfer model shows no consistent benefits.

arXiv comment7 pages, 15 tables, 1 equation