3 papers
cs.AI2026
UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling
Kaiyu Huang, Xingyu Wang, Mingze Kong +6
In real-world deployments of large language models (LLMs), balancing inference quality and computational cost has become a central challenge. Existing approaches tackle this trade-…
cs.DC2026
GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
Boxiao Du, Boning Huangfu, Yizhou Luo +5
Large Language Models (LLMs) play a critical role in emerging agentic applications, where the timely completion of each entire inference is critical. Meanwhile, agentic LLM inferen…
cs.CL2026
AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Serving
Kaiyu Huang, Hao Wu, Zhubo Shi +3
Cloud-based Large Language Model (LLM) services often face challenges in achieving low inference latency and meeting Service Level Objectives (SLOs) under dynamic request patterns.…