The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language Models

Lukas Gienapp, Christopher Schröder, Stefan Schweter +5

Large language model development relies on large-scale training corpora, yet most contain data of unclear licensing status, limiting the development of truly open models. This prob…