4 papers
Probing Factual Knowledge Transfer with Training Data Interventions
Romina Oji, Marc Braun, Marcel Bollmann +2
Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned directly from the target-language d…
Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Training
Lukas Borggren, Jenny Kunz, Marco Kuhlmann
Large language models are increasingly capable in general, but their utility can remain modest in niche or understudied areas. One approach to address this limitation is to special…
Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation
Kevin Glocker, Kätriin Kukk, Romina Oji +3
Achieving high-performing language models which include medium- and lower-resource languages remains a challenge. Massively multilingual models still underperform compared to langu…
Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency
Ehsan Doostmohammadi, Marco Kuhlmann
Retrieval-augmented language models have demonstrated performance comparable to much larger models while requiring fewer computational resources. The effectiveness of these models…