Zyda: A 1.3T Dataset for Open Language Modeling

Yury Tokpanov, Beren Millidge, Paolo Glorioso +4

The size of large language models (LLMs) has scaled dramatically in recent years and their computational and data requirements have surged correspondingly. State-of-the-art languag…