5 papers
Fleet: Hierarchical Task-based Abstraction for Megakernels on Multi-Die GPUs
Sangeeta Chowdhary, Ryan Swann, Sean Siddens +7
Modern GPUs adopt chiplet-based designs with multiple private cache hierarchies, but current programming models (CUDA/HIP) expose a flat execution hierarchy that cannot express chi…
tritonBLAS: Triton-based Analytical Approach for GEMM Kernel Parameter Selection
Ryan Swann, Muhammad Osama, Xiaohu Guo +8
We present tritonBLAS, a fast and deterministic analytical model that uses architectural parameters like the cache hierarchy, and relative code and data placement to generate perfo…
Eliminating Multi-GPU Performance Taxes: A Systems Approach to Efficient Distributed LLMs
Octavian Alexandru Trifan, Karthik Sangaiah, Muhammad Awad +5
As large language models (LLMs) continue to scale, their workloads increasingly rely on distributed execution across multiple GPUs. However, the conventional bulk synchronous paral…
Optimizing Attention on GPUs by Exploiting GPU Architectural NUMA Effects
Mansi Choudhary, Karthik Sangaiah, Sonali Singh +3
The rise of disaggregated AI GPUs has exposed a critical bottleneck in large-scale attention workloads: non-uniform memory access (NUMA). As multi-chiplet designs become the norm f…
SwizzlePerf: Hardware-Aware LLMs for GPU Kernel Performance Optimization
Arya Tschand, Muhammad Awad, Ryan Swann +5
Large language models (LLMs) have shown progress in GPU kernel performance engineering using inefficient search-based methods that optimize around runtime. Any existing approach la…