4 papers
VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU
Zijian He, Adrian Sampson, Yiying Zhang +1
Modern GPUs increasingly rely on specialized and asynchronous hardware units to deliver high performance. Yet these units are often underutilized because today's GPU software stack…
DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference
Shouxu Lin, Zhiyuan Guo, Jiaxin Lin
LLM inference is constrained by GPU memory capacity and bandwidth. Tiered memory architectures mitigate this by allowing the GPU to offload memory to the remote tier. However, exis…
An Early Exploration of Deep-Learning-Driven Prefetching for Far Memory
Yutong Huang, Zhiyuan Guo, Yiying Zhang
Far-memory systems, where applications store less-active data in more energy-efficient memory media, are increasingly adopted by data centers. However, applications are bottlenecke…
Learning Semantics, Not Addresses: Runtime Neural Prefetching for Far Memory
Yutong Huang, Zhiyuan Guo, Yiying Zhang
Memory prefetching has long boosted CPU caches and is increasingly vital for far-memory systems, where large portions of memory are offloaded to cheaper, remote tiers. While effect…