3 papers
cs.DC2026
NCCL M2N: A Layout- and Topology-Aware Collective for Distributed Tensor Resharding
Kaushik Kandadi, Youngeun Kwon, Sreeram Potluri +6
Distributed training and rollout generation often use different tensor layouts, requiring model weights to be resharded across distinct process groups. This M-to-N redistribution i…
cs.DC2026
Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives
Siyuan Shen, Anton Korzh, John Bachan +10
GPU collective communication is typically optimized for bandwidth, yet many emerging workloads are increasingly limited by latency. Long-context decode-heavy large language model (…
cs.GR2021
Cross-layer Visualization and Profiling of Network and I/O Communication for HPC Clusters
Pouya Kousha, Quentin Anthony, Hari Subramoni +1
Understanding and visualizing the full-stack performance trade-offs and interplay between HPC applications, MPI libraries, the communication fabric, and the file system is a challe…