4 papers
Why Do Prefetchers Fail? Let Agents Answer
Xiangfeng Sun, Ceyu Xu, Ningzhi Ai +3
Hardware prefetchers are crucial to processor performance, yet their design remains labor-intensive and expert-driven. Architects inspect execution and memory-access traces, identi…
STS: Efficient Sparse Attention with Speculative Token Sparsity
Ceyu Xu, Jiangnan Yu, Yongji Wu +1
The quadratic complexity of attention imposes severe memory and computational bottlenecks on Large Language Model (LLM) inference. This challenge is particularly acute for emerging…
ICP: Exploiting Instruction Correlation for Prefetching Irregular Memory Accesses
Mengming Li, Chenlu Miao, Buqing Xu +7
Irregular memory accesses pose challenges for effective and efficient data prefetching. While temporal prefetchers have recently shown promise for irregular memory access patterns,…
Hardware-Software Co-design for 3D-DRAM-based LLM Serving Accelerator
Cong Li, Yihan Yin, Chenhao Xue +7
Large language models (LLMs) have been widely deployed for online generative services, where numerous LLM instances jointly handle workloads with fluctuating request arrival rates…