KletterMix: Climbing Toward High-Quality German Pretraining Data - The Full Report

Maurice Kraus, Ruben Härle, Sebastian Sztwiertnia +5

High-quality pretraining data is a central ingredient in modern language models, but German-language resources remain far less developed than their English counterparts: they are o…