InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context

Xin Teng, Canyu Zhang, Shaoyi Zheng +3

Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precomp…