23 citations · 47 across the 13 of their papers we have counts for
13 papers
ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload
Ziyue Liu, Zhengyang Wang, Ruijie Zhang +7
Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare, driving the need for resilient training systems. Yet existing framewor…
Implementing True MPI Sessions and Evaluating MPI Initialization Scalability
Hui Zhou, Kenneth Raffenetti, Yanfei Guo +2
Sessions is one of the major features introduced in the MPI-4 standard. It offers an alternative to the traditional world communicator model by allowing applications to construct c…
ZCCL: Significantly Improving Collective Communication With Error-Bounded Lossy Compression
Jiajun Huang, Sheng Di, Xiaodong Yu +12
With the ever-increasing computing power of supercomputers and the growing scale of scientific applications, the efficiency of MPI collective communication turns out to be a critic…
MPI Progress For All
Hui Zhou, Robert Latham, Ken Raffenetti +2
The progression of communication in the Message Passing Interface (MPI) is not well defined, yet it is critical for application performance, particularly in achieving effective com…
Designing and Prototyping Extensions to MPI in MPICH
Hui Zhou, Ken Raffenetti, Yanfei Guo +3
As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerfu…
Frustrated with MPI+Threads? Try MPIxThreads!
Hui Zhou, Ken Raffenetti, Junchao Zhang +2
MPI+Threads, embodied by the MPI/OpenMP hybrid programming model, is a parallel programming paradigm where threads are used for on-node shared-memory parallelization and MPI is use…
Generating Bindings in MPICH
Hui Zhou, Ken Raffenetti, Wesley Bland +1
The MPI Forum has recently adopted a Python scripting engine for generating the API text in the standard document. As a by-product, it made available reliable and rich descriptions…
MPI Application Binary Interface Standardization
Jeff R. Hammond, Lisandro Dalcin, Erik Schnetter +7
MPI is the most widely used interface for high-performance computing (HPC) workloads. Its success lies in its embrace of libraries and ability to evolve while maintaining backward…
Quantifying the Performance Benefits of Partitioned Communication in MPI
Thomas Gillis, Ken Raffenetti, Hui Zhou +2
Partitioned communication was introduced in MPI 4.0 as a user-friendly interface to support pipelined communication patterns, particularly common in the context of MPI+threads. It…
gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
Jiajun Huang, Sheng Di, Xiaodong Yu +11
GPU-aware collective communication has become a major bottleneck for modern computing platforms as GPU computing power rapidly rises. A traditional approach is to directly integrat…
Accelerating MPI Collectives with Process-in-Process-based Multi-object Techniques
Jiajun Huang, Kaiming Ouyang, Yujia Zhai +8
In the exascale computing era, optimizing MPI collective performance in high-performance computing (HPC) applications is critical. Current algorithms face performance degradation d…
An Optimized Error-controlled MPI Collective Framework Integrated with Lossy Compression
Jiajun Huang, Sheng Di, Xiaodong Yu +11
With the ever-increasing computing power of supercomputers and the growing scale of scientific applications, the efficiency of MPI collective communications turns out to be a criti…
MPIX Stream: An Explicit Solution to Hybrid MPI+X Programming
Hui Zhou, Ken Raffenetti, Yanfei Guo +1
The hybrid MPI+X programming paradigm, where X refers to threads or GPUs, has gained prominence in the high-performance computing arena. This corresponds to a trend of system archi…