An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age

Siddhant Ray, Nick Feamster, Junchen Jiang

LLM inference has become a global-scale, heterogeneous workload spanning agents, retrieval, tool-use, code execution and multi-modal reasoning. These workloads naturally enable con…