most citedMegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

24 citations · 25 across the 4 of their papers we have counts for

collaborators

5 papers

cs.CR2026

Characterizing Statistical Separability in TP-CRIV for Probabilistic AI Models

Teruki Sano, Minoru Kuribayashi, Masao Sakai +4

Third-party challenge-response identity verification (TP-CRIV) enables an independent verifier to assess whether a claimant possesses a model identical to a remotely deployed model…

cs.LG2026

Lossless Anti-Distillation Sampling

Zibo Diao, Jingchu Gai, Xinyue Ai +3

Frontier commercial generative models face a growing threat from distillation, whereby a distiller harvests generated responses and trains a competing model at drastically lower co…

cs.CV2026

Luminark: Training-free, Probabilistically-Certified Watermarking for General Vision Generative Models

Jiayi Xu, Zhang Zhang, Yuanrui Zhang +4

In this paper, we introduce \emph{Luminark}, a training-free and probabilistically-certified watermarking method for general vision generative models. Our approach is built upon a…

cs.DC2024★ 1 cited

Minder: Faulty Machine Detection for Large-scale Distributed Model Training

Yangtao Deng, Xiang Shi, Zhuo Jiang +12

Large-scale distributed model training requires simultaneous training on up to thousands of machines. Faulty machine detection is critical when an unexpected fault occurs in a mach…

cs.LG2024★ 24 cited

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Ziheng Jiang, Haibin Lin, Yinmin Zhong +29

We present the design, implementation and engineering experience in building and deploying MegaScale, a production system for training large language models (LLMs) at the scale of…