StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving

Satyam Kumar, Arpit Singh Gautam, Kailash Talreja +1

Efficient LLM serving must balance throughput and latency across diverse, bursty workloads. We introduce StreamServe, a disaggregated prefill decode serving architecture that combi…