565 citations · 610 across the 78 of their papers we have counts for
24 papers · 1 filter
Derivational Morphology Reveals Analogical Generalization in Large Language Models
Valentin Hofmann, Leonie Weissweiler, David Mortensen +2
What mechanisms underlie linguistic generalization in large language models (LLMs)? This question has attracted considerable attention, with most studies analyzing the extent to wh…
How far can bias go? Tracing bias from pretraining data to alignment
Marion Thaler, Abdullatif Köksal, Alina Leidinger +2
As LLMs are increasingly integrated into user-facing applications, addressing biases that perpetuate societal inequalities is crucial. While much work has gone into measuring or mi…
Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence
Linyang He, Ercong Nie, Helmut Schmid +3
This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) by distinguishing two LLM assessment paradig…
Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization
Mingyang Wang, Lukas Lange, Heike Adel +2
To ensure large language models contain up-to-date knowledge, they need to be updated regularly. However, model editing is challenging as it might also affect knowledge that is unr…
GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages
Amir Hossein Kargaran, François Yvon, Hinrich Schütze
The need for large text corpora has increased with the advent of pretrained language models and, in particular, the discovery of scaling laws for these models. Most available corpo…
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal +3
English-centric large language models (LLMs) often show strong multilingual capabilities. However, their multilingual performance remains unclear and is under-evaluated for many ot…
MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions
Abdullatif Köksal, Marion Thaler, Ayyoob Imani +3
Instruction tuning enhances large language models (LLMs) by aligning them with human preferences across diverse tasks. Traditional approaches to create instruction tuning datasets…
LangSAMP: Language-Script Aware Multilingual Pretraining
Yihong Liu, Haotian Ye, Chunlan Ma +2
Recent multilingual pretrained language models (mPLMs) often avoid using language embeddings -- learnable vectors assigned to individual languages. However, this places a significa…
How Transliterations Improve Crosslingual Alignment
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran +6
Recent studies have shown that post-aligning multilingual pretrained language models (mPLMs) using alignment objectives on both original and transliterated data can improve crossli…
CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
Ingo Ziegler, Abdullatif Köksal, Desmond Elliott +1
Building high-quality datasets for specialized tasks is a time-consuming and resource-intensive process that often requires specialized domain knowledge. We propose Corpus Retrieva…
SYNTHEVAL: Hybrid Behavioral Testing of NLP Models with Synthetic CheckLists
Raoyuan Zhao, Abdullatif Köksal, Yihong Liu +3
Traditional benchmarking in NLP typically involves using static held-out test sets. However, this approach often results in an overestimation of performance and lacks the ability t…
TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
Arda Yüksel, Abdullatif Köksal, Lütfi Kerem Şenel +2
Multiple choice question answering tasks evaluate the reasoning, comprehension, and mathematical abilities of Large Language Models (LLMs). While existing benchmarks employ automat…
Consistent Document-Level Relation Extraction via Counterfactuals
Ali Modarressi, Abdullatif Köksal, Hinrich Schütze
Many datasets have been developed to train and evaluate document-level relation extraction (RE) models. Most of these are constructed using real-world data. It has been shown that…
Learn it or Leave it: Module Composition and Pruning for Continual Learning
Mingyang Wang, Heike Adel, Lukas Lange +2
In real-world environments, continual learning is essential for machine learning models, as they need to acquire new knowledge incrementally without forgetting what they have alrea…
MaskLID: Code-Switching Language Identification through Iterative Masking
Amir Hossein Kargaran, François Yvon, Hinrich Schütze
We present MaskLID, a simple, yet effective, code-switching (CS) language identification (LID) method. MaskLID does not require any training and is designed to complement current h…
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory
Ali Modarressi, Abdullatif Köksal, Ayyoob Imani +2
While current large language models (LLMs) perform well on many knowledge-related tasks, they are limited by relying on their parameters as an implicit storage mechanism. As a resu…
Verbing Weirds Language (Models): Evaluation of English Zero-Derivation in Five LLMs
David R. Mortensen, Valentina Izrailevitch, Yunze Xiao +2
Lexical-syntactic flexibility, in the form of conversion (or zero-derivation) is a hallmark of English morphology. In conversion, a word with one part of speech is placed in a non-…
UCxn: Typologically Informed Annotation of Constructions Atop Universal Dependencies
Leonie Weissweiler, Nina Böbel, Kirian Guiller +11
The Universal Dependencies (UD) project has created an invaluable collection of treebanks with contributions in over 140 languages. However, the UD annotations do not tell the full…
MaiBaam: A Multi-Dialectal Bavarian Universal Dependency Treebank
Verena Blaschke, Barbara Kovačić, Siyao Peng +2
Despite the success of the Universal Dependencies (UD) project exemplified by its impressive language breadth, there is still a lack in `within-language breadth': most treebanks fo…
Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena
Leonie Weissweiler, Abdullatif Köksal, Hinrich Schütze
Argument Structure Constructions (ASCs) are one of the most well-studied construction groups, providing a unique opportunity to demonstrate the usefulness of Construction Grammar (…
Constructions Are So Difficult That Even Large Language Models Get Them Right for the Wrong Reasons
Shijia Zhou, Leonie Weissweiler, Taiqi He +3
In this paper, we make a contribution that can be understood from two perspectives: from an NLP perspective, we introduce a small challenge dataset for NLI with large lexical overl…
Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models
Ercong Nie, Shuzhou Yuan, Bolei Ma +4
Probing the multilingual knowledge of linguistic structure in LLMs, often characterized as sequence labeling, faces challenges with maintaining output templates in current text-to-…
Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
Paul Röttger, Valentin Hofmann, Valentina Pyatkin +4
Much recent work seeks to evaluate values and opinions in large language models (LLMs) using multiple-choice surveys and questionnaires. Most of this work is motivated by concerns…
What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects
Verena Blaschke, Christoph Purschke, Hinrich Schütze +1
Natural language processing (NLP) has largely focused on modelling standardized languages. More recently, attention has increasingly shifted to local, non-standardized languages an…