8 papers · 1 filter
Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training
JiWoo Hwang, Lukas Edman, Alexander Fraser
We investigate two aspects of multilingual pre-training with limited data, namely how language exposure is distributed across English, Dutch, and Chinese, and how that exposure is…
Decomposing and Predicting Annotation Disagreement in Harmful Content Across Languages
Irem Ecem Yelkanat, Faeze Ghorbanpour, Alexander Fraser
Annotator disagreement is increasingly recognized as a meaningful property of subjective tasks, rather than merely noise to be removed. Predicting which items are likely to be cont…
Retrieval Disagreement as a Signal: Diagnosing and Steering Multilingual Dense Embedders
Faeze Ghorbanpour, Shaghayegh Kolli, Jana Diesner +1
Multilingual dense retrievers place all languages in one shared space so that a question finds relevant documents, regardless of which language they are written in. We show that th…
Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Drops
Faeze Ghorbanpour, Constanza Fierro, Alexander Fraser +1
Large Language Models (LLMs) often answer the same factual question differently across languages. We study whether cross-lingual latent-space intervention can reduce this inconsist…
Toppling the Hierarchy in Byte-level Language Modeling
Lukas Edman, Alexander Fraser
This work examines recent byte-level models and their failure to perfectly manipulate characters. State-of-the-art byte-level models use a hierarchical structure, starting at the b…
From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation
Lukas Edman, Daryna Dementieva, Alexander Fraser
Large language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate speech det…
Attribute-Based Diagnosis of LLM Alignment with Hate Speech Annotations
Mohammad Amine Jradi, Faeze Ghorbanpour, Alexander Fraser
Hate speech annotation is costly, subjective, and prone to annotator disagreement, making large-scale dataset construction challenging. We systematically analyze how well large lan…
PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions
Ruhallah Niazi, Faeze Ghorbanpour, Alexander Fraser
Despite impressive multilingual capabilities, large language models (LLMs) remain poorly evaluated on literary knowledge in non-English languages. We introduce PersLitEval, a bench…