7 papers · 1 filter
A Systematic Comparison of Multilingual Interpretability Methods Reveals Anisotropy-Driven Failures
Oskar Holmström, Marcel Bollmann, Marco Kuhlmann
Multilingual language models develop shared cross-lingual representations, and various interpretability methods claim to quantify this sharing. These methods have been developed la…
Probing Factual Knowledge Transfer with Training Data Interventions
Romina Oji, Marc Braun, Marcel Bollmann +2
Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned directly from the target-language d…
Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Training
Lukas Borggren, Jenny Kunz, Marco Kuhlmann
Large language models are increasingly capable in general, but their utility can remain modest in niche or understudied areas. One approach to address this limitation is to special…
Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation
Kevin Glocker, Kätriin Kukk, Romina Oji +3
Achieving high-performing language models which include medium- and lower-resource languages remains a challenge. Massively multilingual models still underperform compared to langu…
Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency
Ehsan Doostmohammadi, Marco Kuhlmann
Retrieval-augmented language models have demonstrated performance comparable to much larger models while requiring fewer computational resources. The effectiveness of these models…
Properties and Challenges of LLM-Generated Explanations
Jenny Kunz, Marco Kuhlmann
The self-rationalising capabilities of large language models (LLMs) have been explored in restricted settings, using task/specific data sets. However, current LLMs do not (only) re…