Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

Cheng Li, Jiexiong Liu, Yixuan Chen +1

On-device LLM inference faces a trilemma of response latency, limited hardware resources and user privacy. Full cloud inference delivers strong computing power but exposes user pro…