5 papers
Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures
Liam Cooper, Shinnung Jeong, Hyeran Jeon +2
Large language model (LLM) outputs are expected to be reproducible under greedy decoding, yet in practice the same model, prompt, and software stack produce different outputs on di…
TileLens: Efficiently Using Large-Granularity Memory Systems with Transparent Two-Dimensional Memory Layout
Jae Hyung Ju, Euijun Chung, Hritvik Taneja +4
Large Language Model (LLM) inference is bottlenecked by the capacity and bandwidth of GPU High-Bandwidth Memory (HBM). Recent proposals, such as High-Bandwidth Flash (HBF) and RoMe…
CuLifter: Lifting GPU Binaries to Typed IR
Jisheng Zhao, Huanzhi Pu, Shinnung Jeong +2
GPU compilers merge all data types into a single unified register file, erasing the type information that binary-analysis tools rely on. We show that type recovery from this untype…
Inside VOLT: Designing an Open-Source GPU Compiler
Shinnung Jeong, Chihyo Ahn, Huanzhi Pu +3
Recent efforts in open-source GPU research are opening new avenues in a domain that has long been tightly coupled with a few commercial vendors. Emerging open GPU architectures def…
Hardware vs. Software Implementation of Warp-Level Features in Vortex RISC-V GPU
Huanzhi Pu, Rishabh Ravi, Shinnung Jeong +5
RISC-V GPUs present a promising path for supporting GPU applications. Traditionally, GPUs achieve high efficiency through the SPMD (Single Program Multiple Data) programming model.…