most citedBreak the Sequential Dependency of LLM Inference Using Lookahead Decoding

3 citations · 6 across the 6 of their papers we have counts for

collaborators
Showing cs.DCShow all

1 paper · 1 filter

cs.DC2024

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving

Jiangfei Duan, Runyu Lu, Haojie Duanmu +5

Large language models (LLMs) have demonstrated remarkable performance, and organizations are racing to serve LLMs of varying sizes as endpoints for use-cases like chat, programming…