3 papers
cs.DC2025
GPU-Initiated Networking for NCCL
Khaled Hamidouche, John Bachan, Pak Markthub +6
Modern AI workloads, especially Mixture-of-Experts (MoE) architectures, increasingly demand low-latency, fine-grained GPU-to-GPU communication with device-side control. Traditional…
cs.DC2025
Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms
Zhiyi Hu, Siyuan Shen, Tommaso Bonato +6
The NVIDIA Collective Communication Library (NCCL) is a critical software layer enabling high-performance collectives on large-scale GPU clusters. Despite being open source with a…
cs.DC2025
PAT: a new algorithm for all-gather and reduce-scatter operations at scale
Sylvain Jeaugey
This paper describes a new algorithm called PAT, for Parallel Aggregated Trees, and which can be used to implement all-gather and reduce-scatter operations. This algorithm works on…