Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
GeoFP8: Geometry-Aware FP8 Gradient Compression for Distributed LLM Training
Jieying Wang, Shuyuan Fan, Zizhong Wang +4
Gradient communication is a primary scaling bottleneck in large language model (LLM) pretraining. Communicating gradients in low-precision formats, such as FP8 and NVFP4, can signi…
cs.DC2025
Pier: Efficient Large Language Model pretraining with Relaxed Global Communication
Shuyuan Fan, Zhao Zhang
Global communication, such as all-reduce and allgather, is the prominent performance bottleneck in large language model (LLM) pretraining. To address this issue, we present Pier, a…