4 papers
On the Linear Speedup of the Push-Pull Method for Decentralized Optimization over Digraphs
Liyuan Liang, Gan Luo, Kun Yuan
The linear speedup property is essential for demonstrating the advantage of distributed algorithms over their single-node counterparts. In this paper, we study the stochastic Push-…
Service-Induced Congestion in Memory-Constrained LLM Serving
Ruicheng Ao, Jing Dong, Gan Luo +1
In large language model (LLM) serving, each request accumulates persistent graphics processing unit (GPU) memory during service as its key-value cache grows with every generated to…
Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
Ruicheng Ao, Gan Luo, David Simchi-Levi +1
Large language models now serve millions of users daily, with providers incurring costs exceeding $700,000 per day. Each request requires token-by-token inference, making GPU sched…
Achieving Linear Speedup and Near-Optimal Complexity for Decentralized Optimization over Row-stochastic Networks
Liyuan Liang, Xinyi Chen, Gan Luo +1
A key challenge in decentralized optimization is determining the optimal convergence rate and designing algorithms to achieve it. While this problem has been extensively addressed…