Showing cs.DCShow all
2 papers · 1 filter
cs.DC2025
Beyond A Single AI Cluster: A Survey of Decentralized LLM Training
Haotian Dong, Jingyan Jiang, Rongwei Lu +5
The emergence of large language models (LLMs) has revolutionized AI development, yet the resource demands beyond a single cluster or even datacenter, limiting accessibility to well…
cs.DC2024
Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
Jiajun Luo, Lizhuo Luo, Jianru Xu +4
Mixture-of-Experts-based (MoE-based) diffusion models demonstrate remarkable scalability in high-fidelity image generation, yet their reliance on expert parallelism introduces crit…