4 citations · 7 across the 10 of their papers we have counts for
11 papers
PCM: Programmable Congestion Management for Terabit RDMA NICs
Mikhail Khalilov, Timo Schneider, Tommaso Bonato +10
Datacenter congestion control (CC) and load balancing (LB) evolve rapidly as workload demands change, but deployment depends on RDMA hardware and transport interfaces. Emerging pro…
Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives
Siyuan Shen, Anton Korzh, John Bachan +10
GPU collective communication is typically optimized for bandwidth, yet many emerging workloads are increasingly limited by latency. Long-context decode-heavy large language model (…
Demystifying NVSHMEM: A System-Level Analysis on Symmetric Memory and Device-Initiated Operations in GPU Communication
Yijun Ma, Siyuan Shen, Tiancheng Chen +6
NVSHMEM is NVIDIA's OpenSHMEM-based PGAS communication library for GPU clusters, enabling GPU-initiated, one-sided communication through symmetric memory. Despite its growing adopt…
EDAN: Towards Understanding Memory Parallelism and Latency Sensitivity in HPC
Siyuan Shen, Mikhail Khalilov, Lukas Gianinazzi +6
Resource disaggregation is a promising technique for improving the efficiency of large-scale computing systems. However, this comes at the cost of increased memory access latency d…
PerfDojo: Automated ML Library Generation for Heterogeneous Architectures
Andrei Ivanov, Siyuan Shen, Gioele Gottardo +5
The increasing complexity of machine learning models and the proliferation of diverse hardware architectures (CPUs, GPUs, accelerators) make achieving optimal performance a signifi…
RailX: A Flexible, Scalable, and Low-Cost Network Architecture for Hyper-Scale LLM Training Systems
Yinxiao Feng, Tiancheng Chen, Yuchen Wei +5
Increasingly large AI workloads are calling for hyper-scale infrastructure; however, traditional interconnection network architecture is neither scalable nor cost-effective enough.…
Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms
Zhiyi Hu, Siyuan Shen, Tommaso Bonato +6
The NVIDIA Collective Communication Library (NCCL) is a critical software layer enabling high-performance collectives on large-scale GPU clusters. Despite being open source with a…
ATLAHS: An Application-centric Network Simulator Toolchain for AI, HPC, and Distributed Storage
Siyuan Shen, Tommaso Bonato, Zhiyi Hu +3
Network simulators play a crucial role in evaluating the performance of large-scale systems. However, existing simulators rely heavily on synthetic microbenchmarks or narrowly focu…
SDR-RDMA: Software-Defined Reliability Architecture for Planetary Scale RDMA Communication
Mikhail Khalilov, Siyuan Shen, Marcin Chrapek +16
RDMA is vital for efficient distributed training across datacenters, but millisecond-scale latencies complicate the design of its reliability layer. We show that depending on long-…
LLAMP: Assessing Network Latency Tolerance of HPC Applications with Linear Programming
Siyuan Shen, Langwen Huang, Marcin Chrapek +5
The shift towards high-bandwidth networks driven by AI workloads in data centers and HPC clusters has unintentionally aggravated network latency, adversely affecting the performanc…
Datacenter Ethernet and RDMA: Issues at Hyperscale
Torsten Hoefler, Duncan Roweth, Keith Underwood +9
We observe that emerging artificial intelligence, high-performance computing, and storage workloads pose new challenges for large-scale datacenter networking. RDMA over Converged E…