Efficient KV Cache Migration for Geo-Distributed LLM Inference in Collaborative Edge Computing

Mingjin Zhang, Jiannong Cao, Tao Wu +2

Large Language Models (LLMs) have shown exceptional capabilities but rely heavily on centralized cloud GPU clusters, leading to high bandwidth costs, latency, and privacy issues. E…