4 papers · 1 filter
A New Lens on Multi-GPU Communication: API-Level Memory Models for Portable Optimization
Johnathan Alsop, Bradford Beckmann, Brandon Potter +5
As network communication increasingly becomes a performance bottleneck for ML and HPC workloads, it is more important than ever to optimize multi-GPU data transfers. Recently, low-…
NUNA: Characterizing and Mitigating Non-Uniform Network Access in Multi-Die GPU Scale-Up Systems
Conor James Green, William Won, Tuan Ta +1
Graphics processing unit (GPU) architectures are growing in size to meet the increasing compute and memory requirements. As GPU sizes increase, intra-socket wire transfer delay inc…
ASTRA-sim 3.0: Next-Level Distributed Machine Learning Simulations via High-Fidelity GPU and Infrastructure Modeling
William Won, Jinsun Yoo, Tuan Ta +16
Distributed machine learning (ML) is a key paradigm for today's large-scale artificial intelligence applications. As model inference arises as an important use case, faithful model…
Analyzing Reverse Address Translation Overheads in Multi-GPU Scale-Up Pods
Amel Fatima, Tuan Ta, Bradford M. Beckmann
Distributed ML workloads rely heavily on collective communication across multi-GPU, multi-node systems. Emerging scale-up fabrics, such as NVLink and UALink, enable direct memory a…