PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks

Huiyou Zhan, Xuan Zhang, Haisheng Tan +4

Large language models (LLMs), while driving a new wave of interactive AI applications across numerous domains, suffer from high inference costs and heavy cloud dependency. Motivate…