1 citations · 1 across the 6 of their papers we have counts for
5 papers · 1 filter
PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
Guotao Yang, Rui Guo, Siwei He +3
Long-running LLM agent workflows often revise interior context spans while retaining long suffixes. Although suffix tokens remain unchanged, altered causal histories and rotary pos…
RAC: Reference-Aware Activation Compression for Communication-Efficient Split LLM Inference
Guotao Yang, Mingxi Zhao, Haopeng Li +4
Large language model (LLM) agents repeatedly process long, privacy-sensitive contexts, while cloud-only deployment exposes user data beyond the trusted endpoint and fully local dep…
AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks
Guotao Yang, Hao Chen, Rui Guo +5
Cloud-edge speculative decoding places a lightweight draft model at an edge gateway and a higher-quality target model in the cloud, but inserts communication into every speculative…
PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
Jinjun Yi, Zhixin Zhao, Yitao Hu +7
LLM serving is increasingly dominated by decode attention, which is a memory-bound operation due to massive KV cache loading from global memory. Meanwhile, real-world workloads exh…
Harpagon: Minimizing DNN Serving Cost via Efficient Dispatching, Scheduling and Splitting
Zhixin Zhao, Yitao Hu, Ziqi Gong +5
Advances in deep neural networks (DNNs) have significantly contributed to the development of real-time video processing applications. Efficient scheduling of DNN workloads in cloud…