4 papers
Towards a Principled Evaluation of Knowledge Editors
Sebastian Pohl, Max Ploner, Alan Akbik
Model editing has been gaining increasing attention over the past few years. For Knowledge Editing in particular, more challenging evaluation datasets have recently been released.…
From Data to Knowledge: Evaluating How Efficiently Language Models Learn Facts
Daniel Christoph, Max Ploner, Patrick Haller +1
Sample efficiency is a crucial property of language models with practical implications for training efficiency. In real-world text, information follows a long-tailed distribution.…
Familiarity: Better Evaluation of Zero-Shot Named Entity Recognition by Quantifying Label Shifts in Synthetic Training Data
Jonas Golde, Patrick Haller, Max Ploner +3
Zero-shot named entity recognition (NER) is the task of detecting named entities of specific types (such as 'Person' or 'Medicine') without any training examples. Current research…
TransformerRanker: A Tool for Efficiently Finding the Best-Suited Language Models for Downstream Classification Tasks
Lukas Garbas, Max Ploner, Alan Akbik
Classification tasks in NLP are typically addressed by selecting a pre-trained language model (PLM) from a model hub, and fine-tuning it for the task at hand. However, given the ve…