7 citations · 21 across the 9 of their papers we have counts for
12 papers · 1 filter
ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload
Ziyue Liu, Zhengyang Wang, Ruijie Zhang +7
Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare, driving the need for resilient training systems. Yet existing framewor…
Implementing True MPI Sessions and Evaluating MPI Initialization Scalability
Hui Zhou, Kenneth Raffenetti, Yanfei Guo +2
Sessions is one of the major features introduced in the MPI-4 standard. It offers an alternative to the traditional world communicator model by allowing applications to construct c…
ZCCL: Significantly Improving Collective Communication With Error-Bounded Lossy Compression
Jiajun Huang, Sheng Di, Xiaodong Yu +12
With the ever-increasing computing power of supercomputers and the growing scale of scientific applications, the efficiency of MPI collective communication turns out to be a critic…
MPI Progress For All
Hui Zhou, Robert Latham, Ken Raffenetti +2
The progression of communication in the Message Passing Interface (MPI) is not well defined, yet it is critical for application performance, particularly in achieving effective com…
Designing and Prototyping Extensions to MPI in MPICH
Hui Zhou, Ken Raffenetti, Yanfei Guo +3
As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerfu…
Frustrated with MPI+Threads? Try MPIxThreads!
Hui Zhou, Ken Raffenetti, Junchao Zhang +2
MPI+Threads, embodied by the MPI/OpenMP hybrid programming model, is a parallel programming paradigm where threads are used for on-node shared-memory parallelization and MPI is use…