ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training

Xueze Kang, Guangyu Xiang, Yuxin Wang +16

Large-scale LLM pretraining now runs across $10^5$--$10^6$ accelerators, making failures routine and elasticity mandatory. We posit that an elastic-native training system must join…