3 papers
cs.DC2026
HyDra: Demystifying and Taming Dynamic Context Parallelism at Production Scale
Zihao Fan, Yunzhuo Liu, Bo Jiang +4
Long-context training runs on sequences whose lengths span orders of magnitude, and dynamic context parallelism (DCP) gives each sequence its own CP degree. Existing DCP systems ei…
cs.DC2026
ARGUS: Production-Scale Tracing and Performance Diagnosis for over 10,000-GPU Clusters
Jiasheng Zhou, Longbin Zeng, Clavis Chen +5
Large-scale LLM training requires always-on, fine-grained observability for effective performance diagnosis at scale. Coarse resource monitors alone cannot localize root causes, an…
cs.OS2025
GoCkpt: Gradient-Assisted Multi-Step overlapped Checkpointing for Efficient LLM Training
Keyao Zhang, Yiquan Chen, Zhuo Hu +3
The accuracy of large language models (LLMs) improves with increasing model size, but increasing model complexity also poses significant challenges to training stability. Periodic…