Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
Mehdi Ali, Manuel Brack, Max Lübbering +15
High-quality multilingual training data is essential for effectively pretraining large language models (LLMs). Yet, the availability of suitable open-source multilingual datasets r…
cs.CL2025
Better Embeddings with Coupled Adam
Felix Stollenwerk, Tobias Stollenwerk
Despite their remarkable capabilities, LLMs learn word representations that exhibit the undesirable yet poorly understood feature of anisotropy. In this paper, we argue that the se…