161 citations · 182 across the 15 of their papers we have counts for
4 papers · 1 filter
TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
Ruiqi Lai, Hongrui Liu, Chengzhi Lu +6
The architectural shift to prefill/decode (PD) disaggregation in LLM serving improves resource utilization but struggles with the bursty nature of modern workloads. Existing autosc…
Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency
Colin Hong, Xu Guo, Anand Chaanan Singh +2
Recently, Test-Time Scaling (TTS) has gained increasing attention for improving LLM reasoning performance at test time without retraining the model. A notable TTS technique is Self…
The High Cost of Keeping Warm: Characterizing Overhead in Serverless Autoscaling Policies
Leonid Kondrashov, Boxi Zhou, Hancheng Wang +1
Serverless computing is transforming cloud application development, but the performance-cost trade-offs of control plane designs remain poorly understood due to a lack of open, cro…
Melding the Serverless Control Plane with the Conventional Cluster Manager for Speed and Resource Efficiency
Leonid Kondrashov, Lazar Cvetković, Hancheng Wang +2
Serverless platforms face a trade-off: conventional cluster managers like Kubernetes offer compatibility for co-locating Function-as-a-Service (FaaS) and Backend-as-a-Service (BaaS…