4 papers · 1 filter
TileLens: Efficiently Using Large-Granularity Memory Systems with Transparent Two-Dimensional Memory Layout
Jae Hyung Ju, Euijun Chung, Hritvik Taneja +4
Large Language Model (LLM) inference is bottlenecked by the capacity and bandwidth of GPU High-Bandwidth Memory (HBM). Recent proposals, such as High-Bandwidth Flash (HBF) and RoMe…
Making Locality-aware GEMM Compatible with Page-Granularity Placement on Chiplet GPUs
Euijun Chung, Jae Hyung Ju, Hyesoon Kim
Multi-chiplet GPUs scale compute throughput and high-bandwidth memory (HBM) capacity, but their non-uniform memory system makes locality between chiplets and their data critical to…
A Fast Locality Simulator for GEMM Design-Space Exploration on Multi-Chiplet GPUs
Euijun Chung, Hyesoon Kim
In multi-chiplet GPUs, memory accesses that cross the silicon interposer to a remote chiplet's high-bandwidth memory (HBM) incur extra latency and energy, making remote-traffic red…
Characterizing CPU-Induced Slowdowns in Multi-GPU LLM Inference
Euijun Chung, Yuxiao Jia, Aaron Jezghani +1
Large-scale machine learning workloads increasingly rely on multi-GPU systems, yet their performance is often limited by an overlooked component: the CPU. Through a detailed study…