DiSCo: Device-Server Collaborative LLM-Based Text Streaming Services

Ting Sun, Penghan Wang, Fan Lai

The rapid rise of large language models (LLMs) in text streaming services has introduced significant cost and Quality of Experience (QoE) challenges in serving millions of daily re…