6 papers
Understanding or Memorizing? A Case Study of German Definite Articles in Language Models
Jonathan Drechsel, Erisa Bytyqi, Steffen Herbold
Language models perform well on grammatical agreement, but it is unclear whether this reflects rule-based generalization or memorization. We study this question for German definite…
GRADIEND: Feature Learning within Neural Networks Exemplified through Biases
Jonathan Drechsel, Steffen Herbold
AI systems frequently exhibit and amplify social biases, leading to harmful consequences in critical areas. This study introduces a novel encoder-decoder approach that leverages mo…
MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training
Jonathan Drechsel, Anja Reusch, Steffen Herbold
Mathematical formulas are a fundamental and widely used component in various scientific fields, serving as a universal language for expressing complex concepts and relationships. W…
SortBench: Benchmarking LLMs based on their ability to sort lists
Steffen Herbold
Sorting is a tedious but simple task for human intelligence and can be solved fairly easily algorithmically. However, for Large Language Models (LLMs) this task is surprisingly har…
Neurosymbolic Architectural Reasoning: Towards Formal Analysis through Neural Software Architecture Inference
Steffen Herbold, Christoph Knieke, Andreas Rausch +1
Formal analysis to ensure adherence of software to defined architectural constraints is not yet broadly used within software development, due to the effort involved in defining for…
From Isolates to Families: Using Neural Networks for Automated Language Affiliation
Frederic Blum, Steffen Herbold, Johann-Mattis List
In historical linguistics, the affiliation of languages to a common language family is traditionally carried out using a complex workflow that relies on manually comparing individu…