HYPIC: Position-Independent Caching for Hybrid-Attention LLM Serving

Yifei Liu, Juntong Wu, Yang Liu +4

In retrieval-augmented generation (RAG) and agentic LLM serving, prompts are assembled from multiple independent segments into long contexts, making prefill dominate the computatio…