CacheBridge: Efficient Cross-Model KV Transfer for Multi-Model LLM Serving

Xingyu Qu, Siyuan Lu, Zhiyu Chen +2

Multi-model LLM serving uses routers, cascades and agent workflows to move requests across models with different costs and capabilities. Existing serving systems reuse KV caches on…