Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN

Tianhang Ding, Jianchun Liu, Hongli Xu

AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: the user attaches to a target…