Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Management

Jiaxun Lu, Xiang Zhang, Yunfeng Shao

Large language model (LLM) inference increasingly spans models that differ in size, capability, price, and provider. This shift creates two costs for developers. One is the integra…