HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching

Yifei Liu, Juntong Wu, Yang Liu +4

In retrieval-augmented generation and agentic LLM serving, prompts are assembled from independent segments into long contexts, making the prefill stage dominate per-request cost. T…