3 papers
cs.DC2026
MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
Juntao Zhao, Qi Lu, Wei Jia +13
Modern frameworks for training large foundation models (LFMs) employ dataloaders in a data-parallel manner, with each loader processing a disjoint subset of training data. When pre…
cs.AI2026
Jenius Agent: Towards Experience-Driven Accuracy Optimization in Real-World Scenarios
Defei Xia, Bingfeng Pi, Shenbin Zhang +3
As agent systems powered by large language models (LLMs) advance, improving performance in context understanding, tool usage, and long-horizon execution has become critical. Howeve…
cs.DC2025
ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUs
Hao Ge, Junda Feng, Qi Huang +6
Scaling long-context ability is essential for Large Language Models (LLMs). To amortize the memory consumption across multiple devices in long-context training, inter-data partitio…