12 citations · 18 across the 16 of their papers we have counts for
15 papers · 1 filter
Repetition over Diversity: High-Signal Data Filtering for Sample-Efficient German Language Modeling
Ansar Aynetdinov, Patrick Haller, Alan Akbik
Recent research has shown that filtering massive English web corpora into high-quality subsets significantly improves training efficiency. However, for high-resource non-English la…
What Matters When Building Universal Multilingual Named Entity Recognition Models?
Jonas Golde, Patrick Haller, Alan Akbik
Recent progress in universal multilingual named entity recognition (NER) has been driven by multilingual transformer models, task-specific architectures, custom loss functions, and…
FiNERweb: Datasets and Artifacts for Scalable Multilingual Named Entity Recognition
Jonas Golde, Patrick Haller, Alan Akbik
Recent multilingual named entity recognition (NER) work has shown that large language models (LLMs) can provide effective synthetic supervision, yet such datasets have mostly appea…
Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements
Patrick Haller, Jonas Golde, Alan Akbik
We study architectural and optimization techniques for sample-efficient language modeling under the constraints of the BabyLM 2025 shared task. Our model, BLaLM, replaces self-atte…
LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance
Patrick Haller, Mark Ibrahim, Polina Kirichenko +2
For Large Language Models (LLMs) to be reliable, they must learn robust knowledge that can be generally applied in diverse settings -- often unlike those seen during training. Yet,…
From Data to Knowledge: Evaluating How Efficiently Language Models Learn Facts
Daniel Christoph, Max Ploner, Patrick Haller +1
Sample efficiency is a crucial property of language models with practical implications for training efficiency. In real-world text, information follows a long-tailed distribution.…
What Matters in Linearizing Language Models? A Comparative Study of Architecture, Scale, and Task Adaptation
Patrick Haller, Jonas Golde, Alan Akbik
Linearization has emerged as a strategy for developing efficient language models (LMs). Starting from an existing Transformer-based LM, linearization replaces the attention compone…
MastermindEval: A Simple But Scalable Reasoning Benchmark
Jonas Golde, Patrick Haller, Fabio Barth +1
Recent advancements in large language models (LLMs) have led to remarkable performance across a wide range of language understanding and mathematical tasks. As a result, increasing…
BabyHGRN: Exploring RNNs for Sample-Efficient Training of Language Models
Patrick Haller, Jonas Golde, Alan Akbik
This paper explores the potential of recurrent neural networks (RNNs) and other subquadratic architectures as competitive alternatives to transformer-based models in low-resource l…
Familiarity: Better Evaluation of Zero-Shot Named Entity Recognition by Quantifying Label Shifts in Synthetic Training Data
Jonas Golde, Patrick Haller, Max Ploner +3
Zero-shot named entity recognition (NER) is the task of detecting named entities of specific types (such as 'Person' or 'Medicine') without any training examples. Current research…
Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilities
Andreas Säuberli, Franz Holzknecht, Patrick Haller +4
Text simplification refers to the process of increasing the comprehensibility of texts. Automatic text simplification models are most commonly evaluated by experts or crowdworkers…
OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMs
Patrick Haller, Ansar Aynetdinov, Alan Akbik
Instruction-tuned Large Language Models (LLMs) have recently showcased remarkable ability to generate fitting responses to natural language instructions. However, an open research…
Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMs
Jonas Golde, Patrick Haller, Felix Hamborg +2
Most NLP tasks are modeled as supervised learning and thus require labeled training data to train effective models. However, manually producing such data at sufficient quality and…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Jason Alan Fries, Leon Weber, Natasha Seelam +40
Training and evaluating language models increasingly requires the construction of meta-datasets --diverse collections of curated data with clear provenance. Natural language prompt…