47 citations · 48 across the 4 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.DC2024
APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving
Yi-Chien Lin, Woosuk Kwon, Ronald Pineda +1
Efficiently serving Large Language Models (LLMs) requires selecting an optimal parallel execution plan, balancing computation, memory, and communication overhead. However, determin…
cs.AI2024
TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput
Xiaoxuan Liu, Jongseok Park, Langxiang Hu +10
Large Language Model (LLM) serving systems batch concurrent user requests to achieve efficient serving. However, in real-world deployments, such inter-request parallelism from batc…