A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference

Yida Zhang, Zhiyong Gao, Shuaibing Yue +2

Recent advancements and widespread adoption of Large Language Models (LLMs) in both industry and academia have catalyzed significant demand for LLM serving. However, traditional cl…