19 citations · 19 across the 3 of their papers we have counts for
3 papers
cs.CL2025
Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
Linda He, Jue Wang, Maurice Weber +3
Large Language Models (LLMs) struggle with long-context reasoning, not only due to the quadratic scaling of computational complexity with sequence length but also because of the sc…
cs.CL2024★ 19 cited
RedPajama: an Open Dataset for Training Large Language Models
Maurice Weber, Daniel Fu, Quentin Anthony +16
Large language models are increasingly becoming a cornerstone technology in artificial intelligence, the sciences, and society as a whole, yet the optimal strategies for dataset co…
cs.CL2024
Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language
Jiayi Wang, Yao Lu, Maurice Weber +4
English, as a very high-resource language, enables the pretraining of high-quality large language models (LLMs). The same cannot be said for most other languages, as leading LLMs s…