2 papers
cs.DC2026
PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
Jinjun Yi, Zhixin Zhao, Yitao Hu +7
LLM serving is increasingly dominated by decode attention, which is a memory-bound operation due to massive KV cache loading from global memory. Meanwhile, real-world workloads exh…
cs.DC2026
PARD: Enhancing Goodput for Inference Pipeline via Proactive Request Dropping
Zhixin Zhao, Yitao Hu, Simin Chen +8
Modern deep neural network (DNN) applications integrate multiple DNN models into inference pipelines with stringent latency requirements for customized tasks. To mitigate extensive…